-- Logs begin at Tue 2022-09-13 19:05:10 UTC, end at Tue 2022-09-13 20:40:08 UTC. -- Sep 13 19:34:14.367521 dv-cinder-828805-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 19:34:14.375768 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 19:34:14 2022] *** Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: nodename: dv-cinder-828805-6-iscsi Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: machine: x86_64 Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: clock source: unix Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: pcre jit disabled Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: detected number of CPU cores: 4 Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: current working directory: / Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: detected binary path: /usr/bin/uwsgi-core Sep 13 19:34:14.378191 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: your processes number limit is 47680 Sep 13 19:34:14.378885 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: your memory page size is 4096 bytes Sep 13 19:34:14.378885 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: detected max file descriptor number: 2048 Sep 13 19:34:14.378885 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: lock engine: pthread robust mutexes Sep 13 19:34:14.378885 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: thunder lock: enabled Sep 13 19:34:14.378885 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 19:34:14.378885 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 19:34:14.417935 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: Python main interpreter initialized at 0x556d03dbb9c0 Sep 13 19:34:14.417935 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: python threads support enabled Sep 13 19:34:14.417935 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: your server socket listen backlog is limited to 100 connections Sep 13 19:34:14.417935 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: your mercy for graceful operations on workers is 90 seconds Sep 13 19:34:14.418323 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 19:34:14.418393 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: *** Operational MODE: preforking *** Sep 13 19:34:14.418648 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: *** uWSGI is running in multiple interpreter mode *** Sep 13 19:34:14.418648 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: spawned uWSGI master process (pid: 88435) Sep 13 19:34:14.418865 dv-cinder-828805-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 19:34:14.419703 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: spawned uWSGI worker 1 (pid: 88441, cores: 1) Sep 13 19:34:14.419873 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: spawned uWSGI worker 2 (pid: 88442, cores: 1) Sep 13 19:34:14.420960 dv-cinder-828805-6-iscsi devstack@keystone.service[88435]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 19:34:18.761633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:34:18.761633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: from cryptography.utils import int_from_bytes Sep 13 19:34:18.762455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:34:18.762455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: from cryptography.utils import int_from_bytes Sep 13 19:34:18.762891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:34:18.762891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: from cryptography.utils import int_from_bytes Sep 13 19:34:18.762999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:34:18.762999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: from cryptography.utils import int_from_bytes Sep 13 19:34:18.814243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 19:34:18.814506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 19:34:18.814506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 19:34:18.814758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 19:34:18.814902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] command line args: [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 19:34:18.815043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] command line args: [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 19:34:18.815191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 19:34:18.815324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 19:34:18.815324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ================================================================================ {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 19:34:18.815554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ================================================================================ {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 19:34:18.815667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] admin_token = **** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.815795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] admin_token = **** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.815901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] config_dir = [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.816020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] config_dir = [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.816134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.816250 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.816358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] config_source = [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.816485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] config_source = [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.816595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] debug = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.816707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] debug = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.816829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.817353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.817353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.817763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.817918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.818072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.818226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.818358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.818521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.818669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.818775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] list_limit = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.818775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] log_config_append = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.818954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] list_limit = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.819081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.819081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] log_dir = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.819375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] log_config_append = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.819537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] log_file = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.819719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.819850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] log_dir = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] log_file = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.820885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.821984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822606 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822853 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.822985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.823276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.823276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] publish_errors = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.823765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.823905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.824048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.824186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.824186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.824186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.824532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.824678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.824818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] publish_errors = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.824948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] standard_threads = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.825141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.825308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.825452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.825667 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.825795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.825989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.825989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] use_journal = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.826236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.826366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] use_json = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.826502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.826629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] use_stderr = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.826629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] use_syslog = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.826937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] standard_threads = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.827066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.827197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.827332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.827463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.827463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.827729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.827871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] use_journal = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.828046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.828739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] use_json = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.828739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] use_stderr = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.829002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.829154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] use_syslog = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.829289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.829417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:34:18.829552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.829707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.829840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.829998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.830133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.830269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.830399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.830531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.830660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.830866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.830995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.831128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.831257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.831396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.831537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.831682 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.831822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.831956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.832089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.832221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.832355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.832499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.832637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.832779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.832912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.833042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.833174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.833318 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.833451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.833580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.833783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.833943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.834142 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.834392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.834531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.834654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.834778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.834905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.835041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.835174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.835304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.835424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.835551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.835678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.835811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.835923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.836053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.836180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.836308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.836440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.836571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.836684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.836795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.836905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.837034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.837154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.837290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.837411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.837529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.837668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.837788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.837904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.838043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.838164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.838294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.838422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.838541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.838665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.838796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.838924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839758 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.839955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.840907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.841913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.842934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.843071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.843180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.843288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.843370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.843489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.843619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.843749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.843874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.844915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.845974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.846091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.846091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.846325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.846450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.846567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.846683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.846800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.846924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847514 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.847984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.848100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.848100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.848315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.848440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.848558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.848674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.848797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.848797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.849016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.849141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.849258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.849374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.849491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.849627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.849747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.849875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.850918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.851040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.851158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.851158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.851454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.851571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.851687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.851804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.851933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.852062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.852207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.852327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.852444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.852560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.852683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.852801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.852926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.853046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.853169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.853291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.853408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.853408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.853668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.853789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.853906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] auth.external = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.854993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.855141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] auth.password = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.855274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.855406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] auth.token = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.855534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.855652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.855770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.855886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] auth.external = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.856965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.857083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.857201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.857320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] credential.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.857437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] auth.password = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.857563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.857698 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] auth.token = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.857829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.857949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.858067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.858191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.858319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.858438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.858555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.858672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.858797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.858922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.859050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.859218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.859343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.859461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.859578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.859695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] credential.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.859821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.859944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.860900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.861120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.861246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.861371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] federation.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.861371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.861593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.861776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.861896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862611 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.862973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.863090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.863207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.863332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.863451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.863580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] federation.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.863697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.863815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.863932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.864058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.864178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.864295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.864412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.864530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.864648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.864774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.864893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.865011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.865128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.865246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.865364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.865555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.865693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.865812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.865929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.866059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.866178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.866305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.866424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.866541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.866659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.866776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.866894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.867981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.868098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.868222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.868341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.868467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.868593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.868711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.868830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.868947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.869064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.869190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.869309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.869427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.869544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.869732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.869854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.869989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.870109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.870233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.870352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.870469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.870587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.870713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.870833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.870952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.871070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.871187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.871305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.871431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.871549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.871674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.871793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.871912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.872028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.872155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.872280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.872399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.872522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.872642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.872760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.872886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.873984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.874102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.874102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.874338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.874455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.874573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.874689 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.875331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.875466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.875586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.875702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.875820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.875937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.876055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.876181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.876300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.876418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.876535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.876658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.876777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.876777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877361 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.877992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.878108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.878227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.878357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.878477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.878594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.878711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.878828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.878945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.879071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.879190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.879308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.879426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.879543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.879661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.879786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.879905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.880974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.881091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.881218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.881338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.881455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.881572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.881572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.881812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.881944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882487 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.882973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.883960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.884068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.884183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.884310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.884514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.884641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.884755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.884871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.885056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.885192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.885309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.885309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.885567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.885823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] resource.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.885983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.886169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.886333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.886333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.886560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.886680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.886798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.886926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.887046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.887164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.887337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.887462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.887581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.887711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.887831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.887949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.888095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.888250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] resource.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.888428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.888625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.888767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] role.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.888951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.889127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] role.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.889301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.889538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.889538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.889825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.890000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.890000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.890325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.890585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.891279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.891450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.891450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.891763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.891902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] role.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] role.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.892974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.893091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.893208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.893325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.893450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.893569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.893765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.893765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.893988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.894106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.894237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.894358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.894475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.894592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.894710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.894827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.894954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.895962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.896183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.896301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.896427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.896547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.896664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.896781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.896781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.896781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.897115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.897234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.897234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] token.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.897453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.897571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.897712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.897832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.897949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.898986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.899111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.899240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.899240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.899462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.899579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.899696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.899822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.899822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.900898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.connection = **** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.901118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.901234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.901360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.901479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.901596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.901735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.901735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.901955 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.902085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.902205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.902325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.902441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] token.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.902558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.902675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.902800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.902919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.903920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.904139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.904265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.904383 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.904501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.904619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.904734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.904851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.904976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88442) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 19:34:18.905731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.connection = **** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.905731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.906406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.906406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.906634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.906855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.907134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.907415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.907695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.907980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.908256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.908539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.908815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.909098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.909372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.909701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.909982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.910268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.910550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:34:18.910762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 19:34:18.935680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.935932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.936120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.937134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.937913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.938142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.945141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.946441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.974109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.974333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.974867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:18.975105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.029151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.029262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.077627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.077860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.079758 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.079995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.233926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.234696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.234894 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.235115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.235203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.235324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.235408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.235536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.235620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.235772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.235994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.236215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.258791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.259955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.260081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.260165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.260345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.271225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.271795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.283864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.284083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.284233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.284360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.284481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.284619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.284741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.284859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.284976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.285094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.285211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.285338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.285458 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.285575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88442) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:34:19.302305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.302552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:34:19.302731 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.302889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:34:19.303046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.303046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:34:19.303330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.303478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.303638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:34:19.303783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.303926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.304069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.304216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.304385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.304560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.304560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.313632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.313930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.314060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.314203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.314327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.314475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.314600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.314854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.321956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.322390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.325104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.325387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.325570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.325752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.325884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.326010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.326170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.326331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.326454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.326585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.326735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.326859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.327001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.327126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.327255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.327375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.327530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.327655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.327800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.327953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.328128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.328371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.351633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.352195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.353132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.353425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.353699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.353985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.358164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.358452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.358626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.358752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.358911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.359709 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.365633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.365927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.366050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.366321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.371813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.372119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.372239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.372411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.372535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.372695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.372815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.372976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.373111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.373231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.373347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.373511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.373669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.373805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.373924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.374067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.390028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.390240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.390432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.390592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.394170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.394318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.397347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.397516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.397677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.397803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.401971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.402251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.405211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.405380 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.408507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.408682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.408818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.409021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.409166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.409290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.409409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.409528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.409666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.409799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.409921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.410039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.426091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.426309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.426492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.426616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.426734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.426872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.426993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.427139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.433829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.434110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.435374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.435681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.436901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.437186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.437341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.438058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.440573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.440804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.440939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.441098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.441221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.441393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.441515 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.441705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.441830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.442100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.453332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.453637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.453839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.454026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.454169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.454324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.454451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.454643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.454841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.454988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.455111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.455275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.455404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.455562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.471473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.471675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.474513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.474781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.477428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.477730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.479042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.479176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.480536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.480682 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.480809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.480930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.481215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.481408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.488199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.488386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.489683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.489828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.492551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.492682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.492948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.493076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.493233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.493357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.493508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.493673 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.501340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.501472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.501657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.501782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.501944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.502082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.508785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.508917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88442) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:34:19.509072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.509195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.509353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.509480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.509643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.509769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.509917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.510042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.510199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.510326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.510481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.510617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.510785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.510907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.511026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.511145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.511285 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.511418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.511576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.511700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.511849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.511973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.512123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.512248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.512375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.512496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88442) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:34:19.557573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556d03dbb9c0 pid: 88442 (default app) Sep 13 19:34:19.557757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x556d03dbb9c0 pid: 88441 (default app) Sep 13 19:34:19.557852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19bc7-aa4d-4508-93c9-a22e149aa4eb None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:19.557852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19bc7-aa4d-4508-93c9-a22e149aa4eb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:19.557852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a19bc7-aa4d-4508-93c9-a22e149aa4eb None None] PATH_INFO: `/v3` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:19.557852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Tue Sep 13 19:34:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:26.428533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5f1af-b272-4b95-a822-77b8be0165f8 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:26.429082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5f1af-b272-4b95-a822-77b8be0165f8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:26.429082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5f1af-b272-4b95-a822-77b8be0165f8 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:26.433637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:26.439391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:26.439943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:26.439943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:26.511493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: INFO dbcounter [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] Registered counter for database keystone Sep 13 19:34:26.531204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG oslo_db.sqlalchemy.engines [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88442) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 19:34:26.537494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writer thread running {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 19:34:26.720702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG passlib.handlers.bcrypt [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88442) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 19:34:26.721011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG passlib.handlers.bcrypt [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88442) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 19:34:26.780251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:26.781966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:26.912754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 19:34:27.364565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5f1af-b272-4b95-a822-77b8be0165f8 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:27.435042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: INFO dbcounter [None req-82d5f1af-b272-4b95-a822-77b8be0165f8 None None] Registered counter for database keystone Sep 13 19:34:27.457573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG oslo_db.sqlalchemy.engines [None req-82d5f1af-b272-4b95-a822-77b8be0165f8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 19:34:27.464031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writer thread running {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 19:34:27.584224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5f1af-b272-4b95-a822-77b8be0165f8 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:27.588574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb1f66c-3fc9-4e20-849b-f4eca3294533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:27.589218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1f66c-3fc9-4e20-849b-f4eca3294533 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:27.589502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1f66c-3fc9-4e20-849b-f4eca3294533 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:27.589804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb1f66c-3fc9-4e20-849b-f4eca3294533 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:27.599419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1f66c-3fc9-4e20-849b-f4eca3294533 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:27.651767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ffb1f66c-3fc9-4e20-849b-f4eca3294533 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 19:34:27.652032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb1f66c-3fc9-4e20-849b-f4eca3294533 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:27.662851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Sep 13 19:34:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:34:27.668814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:27.685533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d41bf0-89e7-49fd-a6d4-24ff56184494 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:27.690476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50339df3-0b80-4fe9-a480-8f23ebf86727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:27.691608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50339df3-0b80-4fe9-a480-8f23ebf86727 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:27.691897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50339df3-0b80-4fe9-a480-8f23ebf86727 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:27.692166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50339df3-0b80-4fe9-a480-8f23ebf86727 admin admin] PATH_INFO: `/v3/projects/61d39e602be54c45a9fb48df499e3bc7` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:27.693786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50339df3-0b80-4fe9-a480-8f23ebf86727 admin admin] RBAC: Authorizing `identity:get_project(project_id=61d39e602be54c45a9fb48df499e3bc7)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:27.746863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING keystone.common.rbac_enforcer.enforcer [None req-50339df3-0b80-4fe9-a480-8f23ebf86727 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 19:34:27.747199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50339df3-0b80-4fe9-a480-8f23ebf86727 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:27.752451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Tue Sep 13 19:34:27 2022] GET /identity/v3/projects/61d39e602be54c45a9fb48df499e3bc7 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:29.891949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89981-102e-4af3-9002-9a4d97a76c92 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:29.892759 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89981-102e-4af3-9002-9a4d97a76c92 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:29.892759 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc89981-102e-4af3-9002-9a4d97a76c92 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:29.896688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:34:29.902813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9e24-018d-442a-a7ad-0c2b131b15fc None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:29.903324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9e24-018d-442a-a7ad-0c2b131b15fc None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:29.903419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a9e24-018d-442a-a7ad-0c2b131b15fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:29.924863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-fc2a9e24-018d-442a-a7ad-0c2b131b15fc None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:29.926427 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fc2a9e24-018d-442a-a7ad-0c2b131b15fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:29.934266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 19:34:30.207304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc89981-102e-4af3-9002-9a4d97a76c92 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:30.219712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc89981-102e-4af3-9002-9a4d97a76c92 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:30.223887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a8ecb-3e7b-4972-b65e-bcc6df208027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:30.224481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a8ecb-3e7b-4972-b65e-bcc6df208027 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:30.224758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a8ecb-3e7b-4972-b65e-bcc6df208027 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:30.225027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a8ecb-3e7b-4972-b65e-bcc6df208027 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:30.232554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a8ecb-3e7b-4972-b65e-bcc6df208027 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:30.234992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27a8ecb-3e7b-4972-b65e-bcc6df208027 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:30.243545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Sep 13 19:34:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:34:30.249281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a9e24-018d-442a-a7ad-0c2b131b15fc None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:30.258411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a9e24-018d-442a-a7ad-0c2b131b15fc None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:30.262761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008bf6e-ac88-4bf8-b93b-b5b8ccdb5d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:30.263419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008bf6e-ac88-4bf8-b93b-b5b8ccdb5d33 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:30.263705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008bf6e-ac88-4bf8-b93b-b5b8ccdb5d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:30.263982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008bf6e-ac88-4bf8-b93b-b5b8ccdb5d33 admin admin] PATH_INFO: `/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:30.265291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008bf6e-ac88-4bf8-b93b-b5b8ccdb5d33 admin admin] RBAC: Authorizing `identity:get_user(user_id=6a5ebebe92dd4d8b9d923b7a40d319f7)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:30.267178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008bf6e-ac88-4bf8-b93b-b5b8ccdb5d33 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:30.268707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Tue Sep 13 19:34:30 2022] GET /identity/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:32.498864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26478-75e7-41aa-856d-6c87b0bfe759 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:32.499434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26478-75e7-41aa-856d-6c87b0bfe759 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:32.499434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26478-75e7-41aa-856d-6c87b0bfe759 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:32.500771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:32.506356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3dd6a-badf-45a7-b82f-5bab1262e576 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:32.506809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3dd6a-badf-45a7-b82f-5bab1262e576 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:32.506998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3dd6a-badf-45a7-b82f-5bab1262e576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:32.531619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-57d3dd6a-badf-45a7-b82f-5bab1262e576 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:32.532821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-57d3dd6a-badf-45a7-b82f-5bab1262e576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:32.533075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7f56-2420-45db-ba44-260e6c747109 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:32.533362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7f56-2420-45db-ba44-260e6c747109 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:32.533660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7f56-2420-45db-ba44-260e6c747109 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:32.535003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:34:32.540546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:32.540843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:32.541104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:32.583916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG passlib.handlers.bcrypt [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88441) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 19:34:32.584297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG passlib.handlers.bcrypt [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88441) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 19:34:32.625943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:32.633591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:32.635245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:32.739660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:32.895264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3dd6a-badf-45a7-b82f-5bab1262e576 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:32.919358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:33.000371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3dd6a-badf-45a7-b82f-5bab1262e576 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:33.011570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:33.036857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eead06b-0fb0-4c51-b532-52ee35a1d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:33.037508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eead06b-0fb0-4c51-b532-52ee35a1d15d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:33.037828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eead06b-0fb0-4c51-b532-52ee35a1d15d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:33.038108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eead06b-0fb0-4c51-b532-52ee35a1d15d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:33.049493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eead06b-0fb0-4c51-b532-52ee35a1d15d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-6eead06b-0fb0-4c51-b532-52ee35a1d15d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:33.058504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:33.059631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:34:33.060670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:34:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:33.066048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eead06b-0fb0-4c51-b532-52ee35a1d15d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:33.116977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f5a93-e8e4-4a1b-93c7-7e286435daa9 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:33.119248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:33.147689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:33.155901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2308832-db0e-4d56-8139-800fcbd9b877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:33.157003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2308832-db0e-4d56-8139-800fcbd9b877 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:33.157284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2308832-db0e-4d56-8139-800fcbd9b877 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:33.157560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2308832-db0e-4d56-8139-800fcbd9b877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:33.165888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eead06b-0fb0-4c51-b532-52ee35a1d15d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:33.200751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:33.201363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:33.201683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:33.201969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:33.203209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:33.203519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:33.204939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:33.216520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:33.222184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:33.223937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2308832-db0e-4d56-8139-800fcbd9b877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:33.225463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2308832-db0e-4d56-8139-800fcbd9b877 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:33.318786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72e3bd-3853-4bcb-ab4d-6cc84b8429a6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:33.322038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:34:33.328441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2308832-db0e-4d56-8139-800fcbd9b877 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:33.331138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:33.352827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761985f-ef62-4855-8fe3-5eab5fc73d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:33.353465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761985f-ef62-4855-8fe3-5eab5fc73d99 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:33.353776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761985f-ef62-4855-8fe3-5eab5fc73d99 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:33.354053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761985f-ef62-4855-8fe3-5eab5fc73d99 None admin] PATH_INFO: `/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:33.355191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761985f-ef62-4855-8fe3-5eab5fc73d99 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5ebebe92dd4d8b9d923b7a40d319f7)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:33.356655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761985f-ef62-4855-8fe3-5eab5fc73d99 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:33.358266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:33 2022] GET /identity/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:33.364172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761985f-ef62-4855-8fe3-5eab5fc73d99 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:33.378533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:33.427531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2308832-db0e-4d56-8139-800fcbd9b877 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:33.462704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:33.462866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761985f-ef62-4855-8fe3-5eab5fc73d99 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:33.463557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:33.463838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:33.464276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:33.483822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:33.485304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:33.498245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdce38-cfd8-4d2a-b2ee-34d6d20ba977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:33.498879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdce38-cfd8-4d2a-b2ee-34d6d20ba977 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:33.499711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdce38-cfd8-4d2a-b2ee-34d6d20ba977 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:33.499988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdce38-cfd8-4d2a-b2ee-34d6d20ba977 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:34:33.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:33.505377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:34:33.506254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:34:33.506254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:34:33.506254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:33.506411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdce38-cfd8-4d2a-b2ee-34d6d20ba977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:33.506762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:34:33.506831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdce38-cfd8-4d2a-b2ee-34d6d20ba977 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:33.508554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:33.513304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:33.513422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdce38-cfd8-4d2a-b2ee-34d6d20ba977 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:33.568844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:33.593810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:33.614022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdce38-cfd8-4d2a-b2ee-34d6d20ba977 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:33.615108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede94a1a-036d-490d-90fa-1135d2ee150c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:33.651743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad46ac-7925-4f66-893a-4ee0b668beea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:33.652362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad46ac-7925-4f66-893a-4ee0b668beea None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:33.652643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad46ac-7925-4f66-893a-4ee0b668beea None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:33.652943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad46ac-7925-4f66-893a-4ee0b668beea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:33.653707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad46ac-7925-4f66-893a-4ee0b668beea None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.domain.id=default, user.id=6a5ebebe92dd4d8b9d923b7a40d319f7) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:33.654024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad46ac-7925-4f66-893a-4ee0b668beea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:33.655528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad46ac-7925-4f66-893a-4ee0b668beea None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:33.661244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07e2ad-7c9e-4642-bf5c-88372263fc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:33.661944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07e2ad-7c9e-4642-bf5c-88372263fc13 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:33.662174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07e2ad-7c9e-4642-bf5c-88372263fc13 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:33.662447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07e2ad-7c9e-4642-bf5c-88372263fc13 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:33.662572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Sep 13 19:34:33 2022] GET /identity/v3/role_assignments?user.id=6a5ebebe92dd4d8b9d923b7a40d319f7&scope.domain.id=default&role.id=b698800ceec3460a8519ea627650da0f => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:33.663099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07e2ad-7c9e-4642-bf5c-88372263fc13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:33.663402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07e2ad-7c9e-4642-bf5c-88372263fc13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:33.664792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07e2ad-7c9e-4642-bf5c-88372263fc13 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:33.685550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:36.058381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cf645-754c-4f0f-90f7-03f9a9c70f0e None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:36.058890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cf645-754c-4f0f-90f7-03f9a9c70f0e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:36.058990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cf645-754c-4f0f-90f7-03f9a9c70f0e None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:36.060270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:36.065906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8275d59-904b-446f-a492-19070ea97830 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:36.066760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8275d59-904b-446f-a492-19070ea97830 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:36.067235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8275d59-904b-446f-a492-19070ea97830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:36.094735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d8275d59-904b-446f-a492-19070ea97830 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:36.096373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d8275d59-904b-446f-a492-19070ea97830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:36.201453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:36.431344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcfc54-4b42-48dc-93db-e3ef9f28709f None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:36.432004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcfc54-4b42-48dc-93db-e3ef9f28709f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:36.432490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcfc54-4b42-48dc-93db-e3ef9f28709f None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:36.434070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:36.441968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362ccc-c8af-4292-bc68-0db88a5e2118 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:36.442780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362ccc-c8af-4292-bc68-0db88a5e2118 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:36.443564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37362ccc-c8af-4292-bc68-0db88a5e2118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:36.468297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-37362ccc-c8af-4292-bc68-0db88a5e2118 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:36.469895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-37362ccc-c8af-4292-bc68-0db88a5e2118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:36.519664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:36.555884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 12/25] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:36.674127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcfc54-4b42-48dc-93db-e3ef9f28709f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:36.822424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcfc54-4b42-48dc-93db-e3ef9f28709f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:36.874970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d82176-5bb7-4cb0-a285-b2c3ae67ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:36.876002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d82176-5bb7-4cb0-a285-b2c3ae67ee82 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:36.876482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d82176-5bb7-4cb0-a285-b2c3ae67ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:36.876978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d82176-5bb7-4cb0-a285-b2c3ae67ee82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:36.889735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d82176-5bb7-4cb0-a285-b2c3ae67ee82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:36.895003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37362ccc-c8af-4292-bc68-0db88a5e2118 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-e0d82176-5bb7-4cb0-a285-b2c3ae67ee82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:36.900127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:36.901004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:34:36.901828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 14/26] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:36.908865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d82176-5bb7-4cb0-a285-b2c3ae67ee82 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:36.917961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:37.068284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37362ccc-c8af-4292-bc68-0db88a5e2118 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:37.119304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c7d3f6-6a7b-4f07-b2d5-46a8d9fffe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:37.119967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7d3f6-6a7b-4f07-b2d5-46a8d9fffe38 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.120256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7d3f6-6a7b-4f07-b2d5-46a8d9fffe38 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.120530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7d3f6-6a7b-4f07-b2d5-46a8d9fffe38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.131951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7d3f6-6a7b-4f07-b2d5-46a8d9fffe38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:37.133461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c7d3f6-6a7b-4f07-b2d5-46a8d9fffe38 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:37.149559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d82176-5bb7-4cb0-a285-b2c3ae67ee82 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-d8c7d3f6-6a7b-4f07-b2d5-46a8d9fffe38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:34:37.166364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:37.167291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:34:37.168106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:34:37.168106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:34:37.168106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:37.168106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 13/27] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:37.168741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163ef94-9e15-4ddc-8e23-ec7fc4ec2033 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.169029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163ef94-9e15-4ddc-8e23-ec7fc4ec2033 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.169303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163ef94-9e15-4ddc-8e23-ec7fc4ec2033 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.170998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 14/28] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:37.177832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163ef94-9e15-4ddc-8e23-ec7fc4ec2033 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:37.190813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c718a38-42b2-459c-9b3b-8861d34d7cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:37.191719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c718a38-42b2-459c-9b3b-8861d34d7cae None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.191994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c718a38-42b2-459c-9b3b-8861d34d7cae None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.192429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c718a38-42b2-459c-9b3b-8861d34d7cae None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.193106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c718a38-42b2-459c-9b3b-8861d34d7cae None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:37.193409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c718a38-42b2-459c-9b3b-8861d34d7cae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:37.194864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c718a38-42b2-459c-9b3b-8861d34d7cae None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:37.205418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 15/29] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:37.207529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406fd3b-ea8d-4382-beb9-2eaa613cfa8a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.207824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406fd3b-ea8d-4382-beb9-2eaa613cfa8a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.208099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406fd3b-ea8d-4382-beb9-2eaa613cfa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.230081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:37.234446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-4406fd3b-ea8d-4382-beb9-2eaa613cfa8a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:37.235685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4406fd3b-ea8d-4382-beb9-2eaa613cfa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:37.320918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163ef94-9e15-4ddc-8e23-ec7fc4ec2033 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:37.332897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:37.345489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 16/30] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:37.347277 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d8df7-55ee-4173-b073-9e9055184d01 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.347554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d8df7-55ee-4173-b073-9e9055184d01 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.347995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d8df7-55ee-4173-b073-9e9055184d01 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.349416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 17/31] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:37.350321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d8df7-55ee-4173-b073-9e9055184d01 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:37.403067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892710c-f057-4e0e-b7a5-2ca0a4786a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:37.404727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892710c-f057-4e0e-b7a5-2ca0a4786a32 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.406370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892710c-f057-4e0e-b7a5-2ca0a4786a32 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.406945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892710c-f057-4e0e-b7a5-2ca0a4786a32 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.408232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892710c-f057-4e0e-b7a5-2ca0a4786a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:37.409675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892710c-f057-4e0e-b7a5-2ca0a4786a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:37.412786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892710c-f057-4e0e-b7a5-2ca0a4786a32 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:37.436974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 15/32] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:37.440451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16ae4-3d68-49b7-a59b-386f1e1591eb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.441885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16ae4-3d68-49b7-a59b-386f1e1591eb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.442350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c16ae4-3d68-49b7-a59b-386f1e1591eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.481465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d1c16ae4-3d68-49b7-a59b-386f1e1591eb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:37.485091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d1c16ae4-3d68-49b7-a59b-386f1e1591eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:37.551200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d8df7-55ee-4173-b073-9e9055184d01 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:37.622780 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff964-e07f-4659-9d5c-ba385dcc4adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:37.624577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff964-e07f-4659-9d5c-ba385dcc4adc None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.624871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff964-e07f-4659-9d5c-ba385dcc4adc None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.625156 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff964-e07f-4659-9d5c-ba385dcc4adc None admin] PATH_INFO: `/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.625984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 16/33] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:37.626411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff964-e07f-4659-9d5c-ba385dcc4adc None admin] RBAC: Authorizing `identity:get_user(user_id=6a5ebebe92dd4d8b9d923b7a40d319f7)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:37.627201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c16ae4-3d68-49b7-a59b-386f1e1591eb None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:37.627974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff964-e07f-4659-9d5c-ba385dcc4adc None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:37.629598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 18/34] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:37 2022] GET /identity/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7 => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:37.630441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff964-e07f-4659-9d5c-ba385dcc4adc None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:37.650002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:37.738206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c16ae4-3d68-49b7-a59b-386f1e1591eb None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:37.779679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f6bac-6ec0-4028-bf43-51e97ba0aafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:37.780404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f6bac-6ec0-4028-bf43-51e97ba0aafa None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.780687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f6bac-6ec0-4028-bf43-51e97ba0aafa None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.780961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476f6bac-6ec0-4028-bf43-51e97ba0aafa None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.781908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f6bac-6ec0-4028-bf43-51e97ba0aafa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:37.786841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476f6bac-6ec0-4028-bf43-51e97ba0aafa None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:37.796971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:37.874471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff964-e07f-4659-9d5c-ba385dcc4adc None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:37.916367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea3402-43e0-47b7-b88a-3a4bb65727b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:37.917026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea3402-43e0-47b7-b88a-3a4bb65727b7 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:37.917317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea3402-43e0-47b7-b88a-3a4bb65727b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:37.917651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea3402-43e0-47b7-b88a-3a4bb65727b7 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:37.918350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea3402-43e0-47b7-b88a-3a4bb65727b7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:37.919801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea3402-43e0-47b7-b88a-3a4bb65727b7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:37.956233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 17/35] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:34:37 2022] POST /identity/v3/projects => generated 301 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:37.957821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f6bac-6ec0-4028-bf43-51e97ba0aafa None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:38.050078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:38.060371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 19/36] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 13 19:34:37 2022] POST /identity/v3/roles => generated 225 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:38.062624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b4baf-41c7-4410-af8b-f81080baec4c None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:38.062922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b4baf-41c7-4410-af8b-f81080baec4c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:38.063802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b4baf-41c7-4410-af8b-f81080baec4c None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:38.064618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 20/37] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:38.070811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e6375-745f-4c5f-b3f2-7f465690dd67 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:38.071260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e6375-745f-4c5f-b3f2-7f465690dd67 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:38.071653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e6375-745f-4c5f-b3f2-7f465690dd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:38.080992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476f6bac-6ec0-4028-bf43-51e97ba0aafa None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:38.098162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-461e6375-745f-4c5f-b3f2-7f465690dd67 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:38.099982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-461e6375-745f-4c5f-b3f2-7f465690dd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:38.131624 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eea76d-e9c8-4e7a-bd3f-56231026439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:38.132558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea76d-e9c8-4e7a-bd3f-56231026439f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:38.132994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea76d-e9c8-4e7a-bd3f-56231026439f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:38.133404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea76d-e9c8-4e7a-bd3f-56231026439f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:38.135914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eea76d-e9c8-4e7a-bd3f-56231026439f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:38.138025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eea76d-e9c8-4e7a-bd3f-56231026439f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:38.140166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 18/38] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:38.141165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eea76d-e9c8-4e7a-bd3f-56231026439f None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:38.209590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 21/39] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:38.210699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e6375-745f-4c5f-b3f2-7f465690dd67 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:38.330377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eea76d-e9c8-4e7a-bd3f-56231026439f None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:38.332686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:38.369109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fc211-bcf7-427e-9dd5-c130486ba408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:38.370047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fc211-bcf7-427e-9dd5-c130486ba408 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:38.370495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fc211-bcf7-427e-9dd5-c130486ba408 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:38.370923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fc211-bcf7-427e-9dd5-c130486ba408 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:38.371774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fc211-bcf7-427e-9dd5-c130486ba408 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:38.373346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fc211-bcf7-427e-9dd5-c130486ba408 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:38.388306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:38.396494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e6375-745f-4c5f-b3f2-7f465690dd67 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:38.398252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 19/40] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 13 19:34:38 2022] POST /identity/v3/roles => generated 219 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:38.399901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38ab2e-d3a2-40be-b6c1-db13e93698cc None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:38.400188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38ab2e-d3a2-40be-b6c1-db13e93698cc None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:38.400469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38ab2e-d3a2-40be-b6c1-db13e93698cc None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:38.402036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 20/41] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:38.408973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859fa6-975b-4704-947e-d15db0245da2 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:38.409264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859fa6-975b-4704-947e-d15db0245da2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:38.409545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2859fa6-975b-4704-947e-d15db0245da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:38.432432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e2859fa6-975b-4704-947e-d15db0245da2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:38.434443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e2859fa6-975b-4704-947e-d15db0245da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:38.442577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffb1f5-4170-463d-942d-fff17c74c10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:38.443694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffb1f5-4170-463d-942d-fff17c74c10a None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:38.444120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffb1f5-4170-463d-942d-fff17c74c10a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:38.444564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffb1f5-4170-463d-942d-fff17c74c10a None admin] PATH_INFO: `/v3/domains/default/users/6a5ebebe92dd4d8b9d923b7a40d319f7/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:38.447689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffb1f5-4170-463d-942d-fff17c74c10a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=6a5ebebe92dd4d8b9d923b7a40d319f7, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:38.450476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffb1f5-4170-463d-942d-fff17c74c10a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:38.469421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 22/42] 10.222.0.23 () {58 vars in 1361 bytes} [Tue Sep 13 19:34:38 2022] PUT /identity/v3/domains/default/users/6a5ebebe92dd4d8b9d923b7a40d319f7/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:34:38.526186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 21/43] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:38.598145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffb1f5-4170-463d-942d-fff17c74c10a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:38.718974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffb1f5-4170-463d-942d-fff17c74c10a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:38.734173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:38.760426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495bd8a-adcb-4007-9902-41ad2d07a31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:38.761042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495bd8a-adcb-4007-9902-41ad2d07a31a None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:38.761322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495bd8a-adcb-4007-9902-41ad2d07a31a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:38.761594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495bd8a-adcb-4007-9902-41ad2d07a31a None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:38.762342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495bd8a-adcb-4007-9902-41ad2d07a31a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:38.763782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495bd8a-adcb-4007-9902-41ad2d07a31a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:38.779846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 23/44] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 13 19:34:38 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:38.888949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2859fa6-975b-4704-947e-d15db0245da2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:39.002921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2859fa6-975b-4704-947e-d15db0245da2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:39.038236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:39.050158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2a645-1fa7-4d9c-b291-3364a0ec0180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:39.050977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2a645-1fa7-4d9c-b291-3364a0ec0180 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:39.051251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2a645-1fa7-4d9c-b291-3364a0ec0180 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:39.051523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2a645-1fa7-4d9c-b291-3364a0ec0180 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:39.052635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2a645-1fa7-4d9c-b291-3364a0ec0180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:39.054176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2a645-1fa7-4d9c-b291-3364a0ec0180 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:39.055758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 22/45] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:39.061983 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495bd8a-adcb-4007-9902-41ad2d07a31a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:39.270087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495bd8a-adcb-4007-9902-41ad2d07a31a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:39.306741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d515db-a854-46a8-8ea5-c2d8746dd0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:39.307358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d515db-a854-46a8-8ea5-c2d8746dd0f1 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:39.307645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d515db-a854-46a8-8ea5-c2d8746dd0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:39.307923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d515db-a854-46a8-8ea5-c2d8746dd0f1 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:39.308590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d515db-a854-46a8-8ea5-c2d8746dd0f1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:39.310092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d515db-a854-46a8-8ea5-c2d8746dd0f1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:39.339915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 24/46] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:34:39 2022] POST /identity/v3/projects => generated 312 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:40.726067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1553bc8-93f9-4ae2-9e81-969e4a316c60 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:40.726590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1553bc8-93f9-4ae2-9e81-969e4a316c60 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:40.726672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1553bc8-93f9-4ae2-9e81-969e4a316c60 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:40.727974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 23/47] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:40.734239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361961c2-0479-4dc3-b0e3-4a660bd70e41 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:40.734676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361961c2-0479-4dc3-b0e3-4a660bd70e41 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:40.734948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361961c2-0479-4dc3-b0e3-4a660bd70e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:40.753129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-361961c2-0479-4dc3-b0e3-4a660bd70e41 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:40.754358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-361961c2-0479-4dc3-b0e3-4a660bd70e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:40.774939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:40.827480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 25/48] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:41.094180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1553bc8-93f9-4ae2-9e81-969e4a316c60 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:41.195160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1553bc8-93f9-4ae2-9e81-969e4a316c60 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:41.235413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa8b6b-a5bf-4959-81ab-f6324ad17598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:41.236058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa8b6b-a5bf-4959-81ab-f6324ad17598 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:41.236358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa8b6b-a5bf-4959-81ab-f6324ad17598 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:41.236623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa8b6b-a5bf-4959-81ab-f6324ad17598 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:41.248180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fa8b6b-a5bf-4959-81ab-f6324ad17598 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-18fa8b6b-a5bf-4959-81ab-f6324ad17598 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:41.255200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:41.256154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:41.256909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 24/49] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:34:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:41.263488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361961c2-0479-4dc3-b0e3-4a660bd70e41 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:41.378997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361961c2-0479-4dc3-b0e3-4a660bd70e41 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:41.418961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa213a-027b-466a-9ba7-041383dfb96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:41.419583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa213a-027b-466a-9ba7-041383dfb96f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:41.419864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa213a-027b-466a-9ba7-041383dfb96f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:41.420137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa213a-027b-466a-9ba7-041383dfb96f None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:41.420841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa213a-027b-466a-9ba7-041383dfb96f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:41.421143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa213a-027b-466a-9ba7-041383dfb96f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:41.422561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa213a-027b-466a-9ba7-041383dfb96f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:41.431086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 26/50] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:41.442556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa8b6b-a5bf-4959-81ab-f6324ad17598 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:41.505717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:41.506635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bfaef-88e2-4830-9908-3fcaf130e5bd None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:41.506999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bfaef-88e2-4830-9908-3fcaf130e5bd None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:41.507283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682bfaef-88e2-4830-9908-3fcaf130e5bd None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:41.508647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 27/51] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:34:41.513834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274a758-6874-4026-b655-452ca5a963bd None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:41.514263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274a758-6874-4026-b655-452ca5a963bd None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:41.515193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274a758-6874-4026-b655-452ca5a963bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:41.535008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e274a758-6874-4026-b655-452ca5a963bd None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:41.536223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e274a758-6874-4026-b655-452ca5a963bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:41.550332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:41.554034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa8b6b-a5bf-4959-81ab-f6324ad17598 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:41.589820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a894f3-912f-425c-b448-05a67ebdb49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:41.590461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a894f3-912f-425c-b448-05a67ebdb49b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:41.590744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a894f3-912f-425c-b448-05a67ebdb49b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:41.591019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a894f3-912f-425c-b448-05a67ebdb49b None admin] PATH_INFO: `/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:41.592543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a894f3-912f-425c-b448-05a67ebdb49b None admin] RBAC: Authorizing `identity:get_user(user_id=6a5ebebe92dd4d8b9d923b7a40d319f7)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:41.594088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a894f3-912f-425c-b448-05a67ebdb49b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:41.595705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 25/52] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:41 2022] GET /identity/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:41.600993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a894f3-912f-425c-b448-05a67ebdb49b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:41.609865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 28/53] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:41.701477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a894f3-912f-425c-b448-05a67ebdb49b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:41.725199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:41.736334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b572f-0e7c-481b-9b34-75a317d22df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:41.736965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b572f-0e7c-481b-9b34-75a317d22df0 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:41.737247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b572f-0e7c-481b-9b34-75a317d22df0 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:41.737519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b572f-0e7c-481b-9b34-75a317d22df0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:41.738680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b572f-0e7c-481b-9b34-75a317d22df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:41.740137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b572f-0e7c-481b-9b34-75a317d22df0 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:41.741722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 26/54] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:41.747484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274a758-6874-4026-b655-452ca5a963bd None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:41.847940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274a758-6874-4026-b655-452ca5a963bd None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:41.880158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b572f-0e7c-481b-9b34-75a317d22df0 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:41.882528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:41.883149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:41.883435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:41.883708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:41.884438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.domain.id=default, user.id=6a5ebebe92dd4d8b9d923b7a40d319f7) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:41.884746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:41.886248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:41.892761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 29/55] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Sep 13 19:34:41 2022] GET /identity/v3/role_assignments?user.id=6a5ebebe92dd4d8b9d923b7a40d319f7&scope.domain.id=default&role.id=b698800ceec3460a8519ea627650da0f => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:42.099230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b572f-0e7c-481b-9b34-75a317d22df0 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:42.134692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9f7e0-95fa-48f1-b773-39792b6e90ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:42.135328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9f7e0-95fa-48f1-b773-39792b6e90ca None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:42.135609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9f7e0-95fa-48f1-b773-39792b6e90ca None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:42.135878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9f7e0-95fa-48f1-b773-39792b6e90ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:42.136975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9f7e0-95fa-48f1-b773-39792b6e90ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:42.138430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b9f7e0-95fa-48f1-b773-39792b6e90ca None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:42.140030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 27/56] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:42.145835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:42.161423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:42.247011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea1fa94-e9f1-45c4-a0c2-69cac30a0bf6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:42.281082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fde1b1-bc5c-4eba-9732-8d6bf6b12d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:42.281636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fde1b1-bc5c-4eba-9732-8d6bf6b12d8d None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:42.281927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fde1b1-bc5c-4eba-9732-8d6bf6b12d8d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:42.282202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fde1b1-bc5c-4eba-9732-8d6bf6b12d8d None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:42.282847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fde1b1-bc5c-4eba-9732-8d6bf6b12d8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:42.284311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fde1b1-bc5c-4eba-9732-8d6bf6b12d8d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:42.310408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 30/57] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Sep 13 19:34:42 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:44.470541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c654c-ea00-4168-aec3-e15fa697117d None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:44.471072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c654c-ea00-4168-aec3-e15fa697117d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:44.471072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c654c-ea00-4168-aec3-e15fa697117d None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:44.472431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 28/58] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:44.477693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158575f2-f8d8-4da4-8e1c-58ce56e4dd19 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:44.478092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158575f2-f8d8-4da4-8e1c-58ce56e4dd19 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:44.478252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158575f2-f8d8-4da4-8e1c-58ce56e4dd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:44.496979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-158575f2-f8d8-4da4-8e1c-58ce56e4dd19 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:44.498210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-158575f2-f8d8-4da4-8e1c-58ce56e4dd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:44.555852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:44.564987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 31/59] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:44.832879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c654c-ea00-4168-aec3-e15fa697117d None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:44.895474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:44.936923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c654c-ea00-4168-aec3-e15fa697117d None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:44.970669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b20f26-fab8-4971-9be3-c85903a7950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:44.971299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20f26-fab8-4971-9be3-c85903a7950f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:44.971579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20f26-fab8-4971-9be3-c85903a7950f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:44.971854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b20f26-fab8-4971-9be3-c85903a7950f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:44.972969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b20f26-fab8-4971-9be3-c85903a7950f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:44.974494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b20f26-fab8-4971-9be3-c85903a7950f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:44.976065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 29/60] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:44.981559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158575f2-f8d8-4da4-8e1c-58ce56e4dd19 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:45.079996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158575f2-f8d8-4da4-8e1c-58ce56e4dd19 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:45.112171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda3037b-fc69-4eab-bb46-f44e29d0ce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:45.112779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3037b-fc69-4eab-bb46-f44e29d0ce82 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:45.113063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3037b-fc69-4eab-bb46-f44e29d0ce82 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:45.113336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda3037b-fc69-4eab-bb46-f44e29d0ce82 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:45.114023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3037b-fc69-4eab-bb46-f44e29d0ce82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:45.115468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda3037b-fc69-4eab-bb46-f44e29d0ce82 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:45.142202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 32/61] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:34:44 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:48.445675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3974f5-9c10-472a-b65c-1d89781e3082 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:48.446234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3974f5-9c10-472a-b65c-1d89781e3082 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:48.446234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3974f5-9c10-472a-b65c-1d89781e3082 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:48.448128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 30/62] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:48.453396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c5244-ea27-4df8-b1cb-f5054538ef20 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:48.453849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c5244-ea27-4df8-b1cb-f5054538ef20 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:48.453988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c5244-ea27-4df8-b1cb-f5054538ef20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:48.460014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90a6a5-63e6-4d58-a341-8ac3429d6a8e None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:48.460556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90a6a5-63e6-4d58-a341-8ac3429d6a8e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:48.461028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90a6a5-63e6-4d58-a341-8ac3429d6a8e None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:48.462596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 31/63] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:34:48.468360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dbca9-1e26-4196-b4d9-17162afaf9ea None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:48.468961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dbca9-1e26-4196-b4d9-17162afaf9ea None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:48.469484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369dbca9-1e26-4196-b4d9-17162afaf9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:48.478610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-361c5244-ea27-4df8-b1cb-f5054538ef20 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:48.479818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-361c5244-ea27-4df8-b1cb-f5054538ef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:48.495138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-369dbca9-1e26-4196-b4d9-17162afaf9ea None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:48.496774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-369dbca9-1e26-4196-b4d9-17162afaf9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:48.575593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 32/64] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:48.578094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:48.618955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 33/65] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:48.907690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c91f8-addf-42d5-8009-d31dccc6f652 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:48.907973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c91f8-addf-42d5-8009-d31dccc6f652 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:48.908401 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c91f8-addf-42d5-8009-d31dccc6f652 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:48.910319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 33/66] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:48.915912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d918b-9546-453b-ab5c-03817e8013ff None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:48.916405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d918b-9546-453b-ab5c-03817e8013ff None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:48.916834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d918b-9546-453b-ab5c-03817e8013ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:48.946802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2f8d918b-9546-453b-ab5c-03817e8013ff None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:48.948609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2f8d918b-9546-453b-ab5c-03817e8013ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:48.957935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77d97d-6ad3-4498-9723-b32c29fd5c59 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:48.958509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77d97d-6ad3-4498-9723-b32c29fd5c59 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:48.959014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77d97d-6ad3-4498-9723-b32c29fd5c59 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:48.960753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 34/67] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:48.965952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a793-ae21-4a15-a6c9-8de34cee9a2d None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:48.966463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a793-ae21-4a15-a6c9-8de34cee9a2d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:48.967483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87a793-ae21-4a15-a6c9-8de34cee9a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:48.988217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4a87a793-ae21-4a15-a6c9-8de34cee9a2d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:48.989753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4a87a793-ae21-4a15-a6c9-8de34cee9a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:49.074949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:49.080643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 34/68] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:49.087151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d918b-9546-453b-ab5c-03817e8013ff None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.090816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 35/69] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:49.112194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a793-ae21-4a15-a6c9-8de34cee9a2d None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.220168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87a793-ae21-4a15-a6c9-8de34cee9a2d None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.249512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d918b-9546-453b-ab5c-03817e8013ff None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.257592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c4ab1-a422-4613-b6d8-4400ab7d3d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.258505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c4ab1-a422-4613-b6d8-4400ab7d3d50 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.258959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c4ab1-a422-4613-b6d8-4400ab7d3d50 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.259405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c4ab1-a422-4613-b6d8-4400ab7d3d50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.270518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c4ab1-a422-4613-b6d8-4400ab7d3d50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:49.272372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-355c4ab1-a422-4613-b6d8-4400ab7d3d50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:49.276993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:49.277962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:34:49.278862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:49.281503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 36/70] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:34:49.283781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648991-3322-45d1-b16f-7199f2748429 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.284219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648991-3322-45d1-b16f-7199f2748429 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.284739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc648991-3322-45d1-b16f-7199f2748429 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.286590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 37/71] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:49.287613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc648991-3322-45d1-b16f-7199f2748429 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.310579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae99ffa-1d8b-4638-8a8a-7d1fbcab0445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.312317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99ffa-1d8b-4638-8a8a-7d1fbcab0445 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.312832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99ffa-1d8b-4638-8a8a-7d1fbcab0445 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.313514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae99ffa-1d8b-4638-8a8a-7d1fbcab0445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.328717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae99ffa-1d8b-4638-8a8a-7d1fbcab0445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-7ae99ffa-1d8b-4638-8a8a-7d1fbcab0445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:49.335434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:49.336270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 35/72] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:49.342344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b9d1-5094-48a0-8f47-f98f0860093b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.343195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b9d1-5094-48a0-8f47-f98f0860093b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.343195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b9d1-5094-48a0-8f47-f98f0860093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.358934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:49.367879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7ce1b9d1-5094-48a0-8f47-f98f0860093b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:49.370059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7ce1b9d1-5094-48a0-8f47-f98f0860093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:49.421499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc648991-3322-45d1-b16f-7199f2748429 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.442138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 36/73] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:49.442789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b9d1-5094-48a0-8f47-f98f0860093b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.457707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.458440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.458723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.459053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.459733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:49.460037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:49.461524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:49.475758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 38/74] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:49.476322 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.550710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:49.554151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b9d1-5094-48a0-8f47-f98f0860093b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.579781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836d549-c2b5-4356-8c8f-5fb553deec84 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.590655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:49.590787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.591354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.591640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.592538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.593220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:49.593528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:49.595114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:49.604415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 37/75] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:49.605237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.622147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d2538-89f1-4b13-bbd6-1eb8e87a2724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.622786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2538-89f1-4b13-bbd6-1eb8e87a2724 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.623093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2538-89f1-4b13-bbd6-1eb8e87a2724 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.623368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d2538-89f1-4b13-bbd6-1eb8e87a2724 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.633975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d2538-89f1-4b13-bbd6-1eb8e87a2724 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-9d3d2538-89f1-4b13-bbd6-1eb8e87a2724 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:49.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:49.639908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:49.640679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 39/76] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:49.641297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d2538-89f1-4b13-bbd6-1eb8e87a2724 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.717800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d0e29-1f2b-4e2d-8e34-3043dbae3867 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.746755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d2538-89f1-4b13-bbd6-1eb8e87a2724 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.752830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83857ec-7a83-4af4-9a5a-5ca43f3204cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.753470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83857ec-7a83-4af4-9a5a-5ca43f3204cc None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.753797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83857ec-7a83-4af4-9a5a-5ca43f3204cc None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.754073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83857ec-7a83-4af4-9a5a-5ca43f3204cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.764228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83857ec-7a83-4af4-9a5a-5ca43f3204cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-e83857ec-7a83-4af4-9a5a-5ca43f3204cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:49.769235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:49.770108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:34:49.770878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 38/77] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:49.771576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83857ec-7a83-4af4-9a5a-5ca43f3204cc None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.782905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd1e7-2c45-4f63-baea-29963aea856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.783538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd1e7-2c45-4f63-baea-29963aea856e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.783818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd1e7-2c45-4f63-baea-29963aea856e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.784092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cd1e7-2c45-4f63-baea-29963aea856e None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.798663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:49.800328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cd1e7-2c45-4f63-baea-29963aea856e None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:49.801833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cd1e7-2c45-4f63-baea-29963aea856e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:49.803486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 40/78] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:49.804246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd1e7-2c45-4f63-baea-29963aea856e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.832285 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:49.874219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83857ec-7a83-4af4-9a5a-5ca43f3204cc None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.905804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cd1e7-2c45-4f63-baea-29963aea856e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:49.908131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21546286-0f7b-4362-9799-80bfa15a73f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.908771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21546286-0f7b-4362-9799-80bfa15a73f2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.909052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21546286-0f7b-4362-9799-80bfa15a73f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.909324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21546286-0f7b-4362-9799-80bfa15a73f2 None admin] PATH_INFO: `/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.910481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21546286-0f7b-4362-9799-80bfa15a73f2 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5ebebe92dd4d8b9d923b7a40d319f7)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:49.911939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21546286-0f7b-4362-9799-80bfa15a73f2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:49.913526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 39/79] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:49.914316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21546286-0f7b-4362-9799-80bfa15a73f2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:49.942652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:49.943270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:49.943554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:49.943828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:49.944497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:49.944802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:49.946218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:49.955656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 41/80] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:49.960766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.004678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:50.017860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21546286-0f7b-4362-9799-80bfa15a73f2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.049048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:50.052926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26ce97-5211-486f-935a-985e4ec8b13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.053565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26ce97-5211-486f-935a-985e4ec8b13d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.053875 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26ce97-5211-486f-935a-985e4ec8b13d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.054148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26ce97-5211-486f-935a-985e4ec8b13d None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.055255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26ce97-5211-486f-935a-985e4ec8b13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.056724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26ce97-5211-486f-935a-985e4ec8b13d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.058326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 40/81] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:50.059037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c02592-8d9d-436b-8e25-40fd259889c9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.062473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26ce97-5211-486f-935a-985e4ec8b13d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.095335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.095947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.096229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.096503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.097178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:50.097473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.098908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.106765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 42/82] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:34:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:50.106765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.168150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26ce97-5211-486f-935a-985e4ec8b13d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.203184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54265518-c66a-4dab-96bd-24d70378452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.203794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54265518-c66a-4dab-96bd-24d70378452b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.204075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54265518-c66a-4dab-96bd-24d70378452b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.204357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54265518-c66a-4dab-96bd-24d70378452b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.209869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8dccfe-f650-4e0e-aa87-fcc2ee018ff8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.232831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:50.243786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13360a-2fc9-4b89-8d5f-fd251e38b9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.244424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13360a-2fc9-4b89-8d5f-fd251e38b9a5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.244711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13360a-2fc9-4b89-8d5f-fd251e38b9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.244987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c13360a-2fc9-4b89-8d5f-fd251e38b9a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.267125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:50.275695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54265518-c66a-4dab-96bd-24d70378452b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.277184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54265518-c66a-4dab-96bd-24d70378452b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.313055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13360a-2fc9-4b89-8d5f-fd251e38b9a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.314598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c13360a-2fc9-4b89-8d5f-fd251e38b9a5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.377009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 41/83] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:34:50.377985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54265518-c66a-4dab-96bd-24d70378452b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.414954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 43/84] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:34:50.415792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13360a-2fc9-4b89-8d5f-fd251e38b9a5 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.445985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:50.473355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:50.488724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54265518-c66a-4dab-96bd-24d70378452b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.520774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c13360a-2fc9-4b89-8d5f-fd251e38b9a5 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.523610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8defdd-c614-43c9-8590-154a3565d7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.524248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8defdd-c614-43c9-8590-154a3565d7d6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.524536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8defdd-c614-43c9-8590-154a3565d7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.524814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8defdd-c614-43c9-8590-154a3565d7d6 None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.526021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8defdd-c614-43c9-8590-154a3565d7d6 None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.527509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8defdd-c614-43c9-8590-154a3565d7d6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.529109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:50 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:50.534282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8defdd-c614-43c9-8590-154a3565d7d6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.553801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127af9f6-3032-433d-b0ab-5b1b7d8a839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.554415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127af9f6-3032-433d-b0ab-5b1b7d8a839b None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.554697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127af9f6-3032-433d-b0ab-5b1b7d8a839b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.555039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127af9f6-3032-433d-b0ab-5b1b7d8a839b None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.555717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127af9f6-3032-433d-b0ab-5b1b7d8a839b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.557147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127af9f6-3032-433d-b0ab-5b1b7d8a839b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.584602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 44/86] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:34:50 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:50.585435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127af9f6-3032-433d-b0ab-5b1b7d8a839b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.643421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8defdd-c614-43c9-8590-154a3565d7d6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.673024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:50.682835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fadd0-9285-47f3-9a41-6247c854b4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.683492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fadd0-9285-47f3-9a41-6247c854b4d7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.683777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fadd0-9285-47f3-9a41-6247c854b4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.684049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fadd0-9285-47f3-9a41-6247c854b4d7 None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.685204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fadd0-9285-47f3-9a41-6247c854b4d7 None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.687285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fadd0-9285-47f3-9a41-6247c854b4d7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.688936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 43/87] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:50 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:50.693989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fadd0-9285-47f3-9a41-6247c854b4d7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.733777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127af9f6-3032-433d-b0ab-5b1b7d8a839b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.747752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:50.778052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51366-fb65-42ee-bfae-34dbe4685d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.778619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51366-fb65-42ee-bfae-34dbe4685d15 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.778908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51366-fb65-42ee-bfae-34dbe4685d15 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.780280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51366-fb65-42ee-bfae-34dbe4685d15 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.780435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf51366-fb65-42ee-bfae-34dbe4685d15 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.781990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf51366-fb65-42ee-bfae-34dbe4685d15 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.784034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 45/88] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:50 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:50.784436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51366-fb65-42ee-bfae-34dbe4685d15 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.826452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fadd0-9285-47f3-9a41-6247c854b4d7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.870680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005675c-6c85-4fe8-9971-cdd465ce1d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.871332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005675c-6c85-4fe8-9971-cdd465ce1d5d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.871617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005675c-6c85-4fe8-9971-cdd465ce1d5d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.871893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005675c-6c85-4fe8-9971-cdd465ce1d5d None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.873740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005675c-6c85-4fe8-9971-cdd465ce1d5d None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:50.875318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005675c-6c85-4fe8-9971-cdd465ce1d5d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:50.877011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 44/89] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:50 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:50.878951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005675c-6c85-4fe8-9971-cdd465ce1d5d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:50.915773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51366-fb65-42ee-bfae-34dbe4685d15 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:50.960634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cd4d3-bf10-42db-977f-49ab2b67bc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:50.961258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cd4d3-bf10-42db-977f-49ab2b67bc66 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:50.961541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cd4d3-bf10-42db-977f-49ab2b67bc66 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:50.961846 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cd4d3-bf10-42db-977f-49ab2b67bc66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:50.966223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:51.017800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005675c-6c85-4fe8-9971-cdd465ce1d5d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:51.023467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:51.055622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cd4d3-bf10-42db-977f-49ab2b67bc66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:51.057117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cd4d3-bf10-42db-977f-49ab2b67bc66 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:51.057496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f6733-116d-4b61-ad7b-dfc78dfdff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:51.058375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6733-116d-4b61-ad7b-dfc78dfdff80 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:51.058887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6733-116d-4b61-ad7b-dfc78dfdff80 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:51.059368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f6733-116d-4b61-ad7b-dfc78dfdff80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:51.139616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f6733-116d-4b61-ad7b-dfc78dfdff80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:51.141474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f6733-116d-4b61-ad7b-dfc78dfdff80 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:51.199814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 46/90] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:34:51.200968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cd4d3-bf10-42db-977f-49ab2b67bc66 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:51.209377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:51.255899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 45/91] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:34:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:34:51.260785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f6733-116d-4b61-ad7b-dfc78dfdff80 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:51.299800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:51.326642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cd4d3-bf10-42db-977f-49ab2b67bc66 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:51.367142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:51.368107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:51.368612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:51.369108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:51.370078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] RBAC: Adding query filter params (role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd, user.id=3678a202c2fa4669a16786db2586e75d) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:51.370594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:51.373065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:51.375037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f6733-116d-4b61-ad7b-dfc78dfdff80 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:51.382952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 47/92] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:34:51 2022] GET /identity/v3/role_assignments?user.id=3678a202c2fa4669a16786db2586e75d&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:51.384017 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:51.449292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:51.449950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:51.450232 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:51.450504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:51.451354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd, user.id=6a5ebebe92dd4d8b9d923b7a40d319f7) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:51.451711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:51.453288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:51.466248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 46/93] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:34:51 2022] GET /identity/v3/role_assignments?user.id=6a5ebebe92dd4d8b9d923b7a40d319f7&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=b698800ceec3460a8519ea627650da0f => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:51.467632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:51.521220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:51.547424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935151a-2644-4b4c-885f-ca3c431f19ac None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:51.618387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbdf87-277c-4716-b1b9-46da96ee4e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:51.619629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbdf87-277c-4716-b1b9-46da96ee4e2b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:51.620153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbdf87-277c-4716-b1b9-46da96ee4e2b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:51.620708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbdf87-277c-4716-b1b9-46da96ee4e2b None admin] PATH_INFO: `/v3/projects/73b78d25934e489f84431200ec41dd2d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:51.626563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbdf87-277c-4716-b1b9-46da96ee4e2b None admin] RBAC: Authorizing `identity:get_project(project_id=73b78d25934e489f84431200ec41dd2d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:51.626888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef13396-c14d-4c58-b8d1-ab875fc38f30 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:51.628955 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbdf87-277c-4716-b1b9-46da96ee4e2b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:51.631344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 48/94] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:51 2022] GET /identity/v3/projects/73b78d25934e489f84431200ec41dd2d => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:51.637635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbdf87-277c-4716-b1b9-46da96ee4e2b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:51.660666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:51.671281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41d930-9b65-4d80-97ad-393e47b41b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:51.673767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d930-9b65-4d80-97ad-393e47b41b13 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:51.674052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d930-9b65-4d80-97ad-393e47b41b13 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:51.674326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41d930-9b65-4d80-97ad-393e47b41b13 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:51.677400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41d930-9b65-4d80-97ad-393e47b41b13 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:51.678099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41d930-9b65-4d80-97ad-393e47b41b13 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:51.681041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 47/95] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:51 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:51.687837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41d930-9b65-4d80-97ad-393e47b41b13 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:51.788089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbdf87-277c-4716-b1b9-46da96ee4e2b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:51.795913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:51.803009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41d930-9b65-4d80-97ad-393e47b41b13 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:51.834187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb58b86-54f2-403e-bc9b-960856090c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:51.835182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb58b86-54f2-403e-bc9b-960856090c89 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:51.835693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb58b86-54f2-403e-bc9b-960856090c89 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:51.836170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb58b86-54f2-403e-bc9b-960856090c89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:51.837283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb58b86-54f2-403e-bc9b-960856090c89 None admin] RBAC: Adding query filter params (role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=73b78d25934e489f84431200ec41dd2d, user.id=3678a202c2fa4669a16786db2586e75d) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:51.837821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb58b86-54f2-403e-bc9b-960856090c89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:51.839725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb58b86-54f2-403e-bc9b-960856090c89 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:51.842065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966cb69-d11f-4d19-9ebb-44e69e976ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:51.842690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966cb69-d11f-4d19-9ebb-44e69e976ee0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:51.842979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966cb69-d11f-4d19-9ebb-44e69e976ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:51.843256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966cb69-d11f-4d19-9ebb-44e69e976ee0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:51.844008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966cb69-d11f-4d19-9ebb-44e69e976ee0 None admin] RBAC: Adding query filter params (role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd, user.id=3678a202c2fa4669a16786db2586e75d) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:51.844319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966cb69-d11f-4d19-9ebb-44e69e976ee0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:51.845892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966cb69-d11f-4d19-9ebb-44e69e976ee0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:51.848482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 49/96] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:34:51 2022] GET /identity/v3/role_assignments?user.id=3678a202c2fa4669a16786db2586e75d&scope.project.id=73b78d25934e489f84431200ec41dd2d&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:51.857645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 48/97] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:34:51 2022] GET /identity/v3/role_assignments?user.id=3678a202c2fa4669a16786db2586e75d&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=90097e8ce1394daba58b200de808ad59 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:53.621430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa9950-2d24-4439-a42d-1d3786196f37 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:53.622554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa9950-2d24-4439-a42d-1d3786196f37 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:53.623898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa9950-2d24-4439-a42d-1d3786196f37 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:53.625893 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 50/98] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:53.631851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc8c0a-20f6-47d5-bf8d-fb0596ce2904 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:53.632349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc8c0a-20f6-47d5-bf8d-fb0596ce2904 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:53.632421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc8c0a-20f6-47d5-bf8d-fb0596ce2904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:53.663466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-58bc8c0a-20f6-47d5-bf8d-fb0596ce2904 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:53.664685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-58bc8c0a-20f6-47d5-bf8d-fb0596ce2904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:53.742886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 49/99] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:53.900050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d0eac-f27a-41d8-9c33-6b7aaa4620f0 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:53.900339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d0eac-f27a-41d8-9c33-6b7aaa4620f0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:53.900590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d0eac-f27a-41d8-9c33-6b7aaa4620f0 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:53.903203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 51/100] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:53.918419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26f8b-f3de-4c32-94bf-ef41b6fed036 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:53.918726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26f8b-f3de-4c32-94bf-ef41b6fed036 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:53.918983 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26f8b-f3de-4c32-94bf-ef41b6fed036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:53.947049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-faa26f8b-f3de-4c32-94bf-ef41b6fed036 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:53.948556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-faa26f8b-f3de-4c32-94bf-ef41b6fed036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:54.043591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 50/101] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:54.216422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812c32d-aa3c-411b-9ae1-12bc20ae7031 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:54.216752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812c32d-aa3c-411b-9ae1-12bc20ae7031 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:54.216935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812c32d-aa3c-411b-9ae1-12bc20ae7031 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:54.218715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 52/102] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:54.224992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b772f-a76d-4360-ba8b-6ff9a9844437 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:54.225395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b772f-a76d-4360-ba8b-6ff9a9844437 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:54.225704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b772f-a76d-4360-ba8b-6ff9a9844437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:54.258877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-790b772f-a76d-4360-ba8b-6ff9a9844437 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:54.260080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-790b772f-a76d-4360-ba8b-6ff9a9844437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:54.279794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:54.344266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 51/103] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:54.376956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812c32d-aa3c-411b-9ae1-12bc20ae7031 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:54.458943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:54.468130 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b772f-a76d-4360-ba8b-6ff9a9844437 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:54.519397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812c32d-aa3c-411b-9ae1-12bc20ae7031 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:54.581676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b772f-a76d-4360-ba8b-6ff9a9844437 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:54.619126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b4a5-733d-455a-92ce-478c0e0f1b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:54.619865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b4a5-733d-455a-92ce-478c0e0f1b04 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:54.620218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b4a5-733d-455a-92ce-478c0e0f1b04 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:54.620557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b4a5-733d-455a-92ce-478c0e0f1b04 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:54.626342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ece042-04ed-402d-bafd-c7b246eeea95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:54.627284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece042-04ed-402d-bafd-c7b246eeea95 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:54.627568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece042-04ed-402d-bafd-c7b246eeea95 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:54.627848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ece042-04ed-402d-bafd-c7b246eeea95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:54.629025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ece042-04ed-402d-bafd-c7b246eeea95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:54.630549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ece042-04ed-402d-bafd-c7b246eeea95 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:54.632382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 53/104] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:54.633565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72b4a5-733d-455a-92ce-478c0e0f1b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:54.637595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ece042-04ed-402d-bafd-c7b246eeea95 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-8e72b4a5-733d-455a-92ce-478c0e0f1b04 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:54.639499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:54.642082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:34:54.644078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 52/105] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:34:54.650965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b4a5-733d-455a-92ce-478c0e0f1b04 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:54.666008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:54.760529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b4a5-733d-455a-92ce-478c0e0f1b04 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:54.797420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:54.798088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:54.798370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:54.798646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:54.799414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:54.799725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:54.801230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:54.810009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 53/106] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:34:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:54.815895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:54.820259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ece042-04ed-402d-bafd-c7b246eeea95 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:54.886055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c81d-f19d-40a8-9fb6-3f52630892e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:54.886725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c81d-f19d-40a8-9fb6-3f52630892e9 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:54.887016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c81d-f19d-40a8-9fb6-3f52630892e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:54.887290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32c81d-f19d-40a8-9fb6-3f52630892e9 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:54.887968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32c81d-f19d-40a8-9fb6-3f52630892e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:54.888523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:54.889485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d32c81d-f19d-40a8-9fb6-3f52630892e9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:54.915426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:54.930355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e6df6-1518-4840-8607-ab371ed68cb3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:54.937819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 54/107] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:34:54 2022] POST /identity/v3/projects => generated 302 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:34:54.938719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c81d-f19d-40a8-9fb6-3f52630892e9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:55.012713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da51da-9de2-4442-bf4a-c675bc224078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:55.012713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da51da-9de2-4442-bf4a-c675bc224078 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.012713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da51da-9de2-4442-bf4a-c675bc224078 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.012713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da51da-9de2-4442-bf4a-c675bc224078 None admin] PATH_INFO: `/v3/users/7c1dff78994d4e459ebcedaf94641bea` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.043493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da51da-9de2-4442-bf4a-c675bc224078 None admin] RBAC: Authorizing `identity:get_user(user_id=7c1dff78994d4e459ebcedaf94641bea)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:55.045531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da51da-9de2-4442-bf4a-c675bc224078 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:55.049541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 54/108] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:54 2022] GET /identity/v3/users/7c1dff78994d4e459ebcedaf94641bea => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:34:55.055533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da51da-9de2-4442-bf4a-c675bc224078 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:55.165540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da51da-9de2-4442-bf4a-c675bc224078 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:55.189208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32c81d-f19d-40a8-9fb6-3f52630892e9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:55.203253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd42450-06ea-4e1e-9559-8389b886ddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:55.203924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd42450-06ea-4e1e-9559-8389b886ddcc None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.204211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd42450-06ea-4e1e-9559-8389b886ddcc None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.204484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd42450-06ea-4e1e-9559-8389b886ddcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.214124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:55.277581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75149d9a-d642-4fae-bd49-6c3037420f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:55.278201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149d9a-d642-4fae-bd49-6c3037420f2d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.278487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149d9a-d642-4fae-bd49-6c3037420f2d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.278771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149d9a-d642-4fae-bd49-6c3037420f2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.281058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd42450-06ea-4e1e-9559-8389b886ddcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:55.282921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd42450-06ea-4e1e-9559-8389b886ddcc None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:55.294256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75149d9a-d642-4fae-bd49-6c3037420f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-75149d9a-d642-4fae-bd49-6c3037420f2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:55.301337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:55.302374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:55.303133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 55/109] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:34:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:55.309695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bb4b-ff72-46f3-b8d1-3b6901a0d20f None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.310003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bb4b-ff72-46f3-b8d1-3b6901a0d20f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.310331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bb4b-ff72-46f3-b8d1-3b6901a0d20f None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.311879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 56/110] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:55.314212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687bb4b-ff72-46f3-b8d1-3b6901a0d20f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:55.408039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 55/111] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:34:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:34:55.408696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:55.409846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d0660-f51c-4b75-9ee0-c5aed4d2a977 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.410144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d0660-f51c-4b75-9ee0-c5aed4d2a977 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.410470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d0660-f51c-4b75-9ee0-c5aed4d2a977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.439487 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-127d0660-f51c-4b75-9ee0-c5aed4d2a977 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:55.441672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-127d0660-f51c-4b75-9ee0-c5aed4d2a977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:55.456979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687bb4b-ff72-46f3-b8d1-3b6901a0d20f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:55.473966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:55.498919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:55.499710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.499994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.500317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.501006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:55.501318 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:55.502922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:55.512295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 57/112] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:55.517762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:55.532110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 56/113] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:55.537648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e602b6-7d79-4f3f-86ca-556d7529b045 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.537648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e602b6-7d79-4f3f-86ca-556d7529b045 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.537648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e602b6-7d79-4f3f-86ca-556d7529b045 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.537648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 57/114] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:55.537648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e602b6-7d79-4f3f-86ca-556d7529b045 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:55.642015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba7621-17f8-43cf-9e4e-f078d6fd76da None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:55.662124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:55.674546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e602b6-7d79-4f3f-86ca-556d7529b045 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:55.690547 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf6f33-8ff9-401c-863a-9d9ae362e865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:55.691601 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6f33-8ff9-401c-863a-9d9ae362e865 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.692030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6f33-8ff9-401c-863a-9d9ae362e865 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.692439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6f33-8ff9-401c-863a-9d9ae362e865 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.693642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf6f33-8ff9-401c-863a-9d9ae362e865 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:55.695778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf6f33-8ff9-401c-863a-9d9ae362e865 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:55.697444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 58/115] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:55 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:55.698856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9d941-8777-4371-9613-a0a484be009f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.699142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9d941-8777-4371-9613-a0a484be009f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.699976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9d941-8777-4371-9613-a0a484be009f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.715366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254a4dd-72dd-461d-8863-5fb0fdca4690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:55.716309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254a4dd-72dd-461d-8863-5fb0fdca4690 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.717317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254a4dd-72dd-461d-8863-5fb0fdca4690 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.717850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254a4dd-72dd-461d-8863-5fb0fdca4690 None admin] PATH_INFO: `/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.719222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254a4dd-72dd-461d-8863-5fb0fdca4690 None admin] RBAC: Authorizing `identity:get_user(user_id=6a5ebebe92dd4d8b9d923b7a40d319f7)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:55.721148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254a4dd-72dd-461d-8863-5fb0fdca4690 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:55.722760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d2a9d941-8777-4371-9613-a0a484be009f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:55.723319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 58/116] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:55 2022] GET /identity/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:55.723985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d2a9d941-8777-4371-9613-a0a484be009f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:55.725071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd4ab-50d2-4795-b7e6-f036c90e65c9 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.725555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd4ab-50d2-4795-b7e6-f036c90e65c9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.726046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd4ab-50d2-4795-b7e6-f036c90e65c9 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.727492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 59/117] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:55.728491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dd4ab-50d2-4795-b7e6-f036c90e65c9 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:55.765121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:55.818117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 59/118] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:55.818117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9d941-8777-4371-9613-a0a484be009f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:55.869673 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dd4ab-50d2-4795-b7e6-f036c90e65c9 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:55.916067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199046b-713e-4ceb-96e5-a4ae1ce43839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:55.917145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199046b-713e-4ceb-96e5-a4ae1ce43839 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.917444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199046b-713e-4ceb-96e5-a4ae1ce43839 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.917759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199046b-713e-4ceb-96e5-a4ae1ce43839 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.918847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199046b-713e-4ceb-96e5-a4ae1ce43839 None admin] RBAC: Adding query filter params (role.id=935445ed062541ceb33d5021be2b092a, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd, user.id=7c1dff78994d4e459ebcedaf94641bea) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:55.919171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199046b-713e-4ceb-96e5-a4ae1ce43839 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:55.921954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199046b-713e-4ceb-96e5-a4ae1ce43839 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:55.933790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 60/119] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:34:55 2022] GET /identity/v3/role_assignments?user.id=7c1dff78994d4e459ebcedaf94641bea&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=935445ed062541ceb33d5021be2b092a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:55.934997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd4266-250c-405d-9994-83a85739591c None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:55.935412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd4266-250c-405d-9994-83a85739591c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:55.938283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd4266-250c-405d-9994-83a85739591c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:55.938440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9d941-8777-4371-9613-a0a484be009f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:55.965259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:55.971465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-95bd4266-250c-405d-9994-83a85739591c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:55.973255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-95bd4266-250c-405d-9994-83a85739591c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:56.027063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654e536-0e44-4925-9098-cf401580be9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:56.027728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654e536-0e44-4925-9098-cf401580be9c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:56.028009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654e536-0e44-4925-9098-cf401580be9c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:56.028279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654e536-0e44-4925-9098-cf401580be9c None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:56.034452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654e536-0e44-4925-9098-cf401580be9c None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:56.035946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654e536-0e44-4925-9098-cf401580be9c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:56.037996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 60/120] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:55 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:56.038793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654e536-0e44-4925-9098-cf401580be9c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:56.089831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 61/121] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:56.090694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd4266-250c-405d-9994-83a85739591c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:56.145665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:56.204787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654e536-0e44-4925-9098-cf401580be9c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:56.263485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3e4d6-3f9c-4299-b94d-c8c02321b032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:56.264143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3e4d6-3f9c-4299-b94d-c8c02321b032 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:56.264433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3e4d6-3f9c-4299-b94d-c8c02321b032 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:56.264721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3e4d6-3f9c-4299-b94d-c8c02321b032 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:56.274385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd4266-250c-405d-9994-83a85739591c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:56.288407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d3e4d6-3f9c-4299-b94d-c8c02321b032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-c5d3e4d6-3f9c-4299-b94d-c8c02321b032 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:56.297093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:56.297996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:34:56.298776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:56.300064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 61/122] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:56.305099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3e4d6-3f9c-4299-b94d-c8c02321b032 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:56.349411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b1d1f-1271-4b10-9c7e-567448e0b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:56.350034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1d1f-1271-4b10-9c7e-567448e0b847 None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:56.350321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1d1f-1271-4b10-9c7e-567448e0b847 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:56.350602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5b1d1f-1271-4b10-9c7e-567448e0b847 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/users/6a5ebebe92dd4d8b9d923b7a40d319f7/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:56.352547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b1d1f-1271-4b10-9c7e-567448e0b847 None admin] RBAC: Authorizing `identity:create_grant(project_id=485f0f7bdc174746a34e4c85b269d3dd, user_id=6a5ebebe92dd4d8b9d923b7a40d319f7, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:56.354461 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5b1d1f-1271-4b10-9c7e-567448e0b847 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:56.374876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 62/123] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:34:56 2022] PUT /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/users/6a5ebebe92dd4d8b9d923b7a40d319f7/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:34:56.376805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b1d1f-1271-4b10-9c7e-567448e0b847 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:56.416875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:56.490659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:56.540349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5b1d1f-1271-4b10-9c7e-567448e0b847 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:56.580525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3e4d6-3f9c-4299-b94d-c8c02321b032 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:56.596442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:56.597078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:56.597358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:56.597669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:56.598388 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:56.598660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:56.600198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:56.616883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 63/124] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:56.617185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:56.624390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205350c7-f387-4e32-8dc9-0f71310c2a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:56.625356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205350c7-f387-4e32-8dc9-0f71310c2a80 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:56.625863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205350c7-f387-4e32-8dc9-0f71310c2a80 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:56.626331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205350c7-f387-4e32-8dc9-0f71310c2a80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:56.642847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205350c7-f387-4e32-8dc9-0f71310c2a80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-205350c7-f387-4e32-8dc9-0f71310c2a80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:56.647358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:56.648199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:34:56.648959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:56.652219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 62/125] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:56.659207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205350c7-f387-4e32-8dc9-0f71310c2a80 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:56.773322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc958c-cc7f-41af-806f-8aab25d3ef81 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:56.815555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380e5fb-c9bd-4c4e-ac8d-e1257f16ef4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:56.816206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380e5fb-c9bd-4c4e-ac8d-e1257f16ef4b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:56.816496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380e5fb-c9bd-4c4e-ac8d-e1257f16ef4b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:56.816777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380e5fb-c9bd-4c4e-ac8d-e1257f16ef4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:56.830622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380e5fb-c9bd-4c4e-ac8d-e1257f16ef4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-c380e5fb-c9bd-4c4e-ac8d-e1257f16ef4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:56.836577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:56.837457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:34:56.838285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 64/126] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:34:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:56.839454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380e5fb-c9bd-4c4e-ac8d-e1257f16ef4b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:56.842900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:56.867086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:56.870995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205350c7-f387-4e32-8dc9-0f71310c2a80 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:56.910566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088999a-e965-4f6c-a30b-485049794fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:56.911571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088999a-e965-4f6c-a30b-485049794fc1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:56.912057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088999a-e965-4f6c-a30b-485049794fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:56.912531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088999a-e965-4f6c-a30b-485049794fc1 None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:56.913949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088999a-e965-4f6c-a30b-485049794fc1 None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:56.915652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088999a-e965-4f6c-a30b-485049794fc1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:56.917470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 63/127] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:56 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:56.922581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088999a-e965-4f6c-a30b-485049794fc1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.044918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380e5fb-c9bd-4c4e-ac8d-e1257f16ef4b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:57.070676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088999a-e965-4f6c-a30b-485049794fc1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:57.110349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:57.111333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.111816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.112291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.113180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:57.113778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:57.115374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:57.124242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 64/128] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:34:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:57.125512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.127047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:57.127678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.127962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.128236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.128909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:57.129214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:57.131382 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:57.152344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 65/129] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:34:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:57.153290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.160948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:57.262503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82753f-a8aa-451d-9694-9aa6090e07e1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:57.283077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:57.300202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c375674-1f7d-4db8-a9ce-7be467e85edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:57.302003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375674-1f7d-4db8-a9ce-7be467e85edc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.302500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375674-1f7d-4db8-a9ce-7be467e85edc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.302998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375674-1f7d-4db8-a9ce-7be467e85edc None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.304406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c375674-1f7d-4db8-a9ce-7be467e85edc None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:57.306149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c375674-1f7d-4db8-a9ce-7be467e85edc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:57.308509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 65/130] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:57 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:57.315238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c375674-1f7d-4db8-a9ce-7be467e85edc None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.374705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897737e1-2ea4-41bb-af5b-304c0b33b32a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:57.428897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:57.432529 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c375674-1f7d-4db8-a9ce-7be467e85edc None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:57.444233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46867e86-b14e-4036-9abf-2a67c212517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:57.445040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46867e86-b14e-4036-9abf-2a67c212517a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.445307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46867e86-b14e-4036-9abf-2a67c212517a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.445580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46867e86-b14e-4036-9abf-2a67c212517a None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.449684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46867e86-b14e-4036-9abf-2a67c212517a None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:57.452511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46867e86-b14e-4036-9abf-2a67c212517a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:57.454427 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 66/131] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:57 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:57.455189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46867e86-b14e-4036-9abf-2a67c212517a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.474267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb272eb-7ce3-41c5-8482-e6a8eac14e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:57.475274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb272eb-7ce3-41c5-8482-e6a8eac14e49 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.475781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb272eb-7ce3-41c5-8482-e6a8eac14e49 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.476257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb272eb-7ce3-41c5-8482-e6a8eac14e49 None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.477736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb272eb-7ce3-41c5-8482-e6a8eac14e49 None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:57.479453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb272eb-7ce3-41c5-8482-e6a8eac14e49 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:57.482707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 66/132] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:57 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:57.484135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb272eb-7ce3-41c5-8482-e6a8eac14e49 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.601335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb272eb-7ce3-41c5-8482-e6a8eac14e49 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:57.648553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e026e-08b5-4166-9baf-ca48b3fd4fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:57.650745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e026e-08b5-4166-9baf-ca48b3fd4fe6 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.652553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e026e-08b5-4166-9baf-ca48b3fd4fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.653236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e026e-08b5-4166-9baf-ca48b3fd4fe6 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.656357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e026e-08b5-4166-9baf-ca48b3fd4fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:57.661638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e026e-08b5-4166-9baf-ca48b3fd4fe6 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:57.665924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 67/133] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:57 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:57.668392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46867e86-b14e-4036-9abf-2a67c212517a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:57.669423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e026e-08b5-4166-9baf-ca48b3fd4fe6 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.708752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:57.715559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade1749-7fd5-409c-87e0-7d5253301816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:57.717599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1749-7fd5-409c-87e0-7d5253301816 None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.718389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1749-7fd5-409c-87e0-7d5253301816 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.718858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade1749-7fd5-409c-87e0-7d5253301816 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/users/3678a202c2fa4669a16786db2586e75d/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.722188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade1749-7fd5-409c-87e0-7d5253301816 None admin] RBAC: Authorizing `identity:create_grant(project_id=485f0f7bdc174746a34e4c85b269d3dd, user_id=3678a202c2fa4669a16786db2586e75d, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:57.724190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade1749-7fd5-409c-87e0-7d5253301816 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:57.730762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:57.744478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 67/134] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:34:57 2022] PUT /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/users/3678a202c2fa4669a16786db2586e75d/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:34:57.751454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade1749-7fd5-409c-87e0-7d5253301816 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.866788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e026e-08b5-4166-9baf-ca48b3fd4fe6 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:57.950824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61f38e-6c16-4487-8fc7-17478a75a166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:57.952063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f38e-6c16-4487-8fc7-17478a75a166 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.955010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f38e-6c16-4487-8fc7-17478a75a166 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.955559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61f38e-6c16-4487-8fc7-17478a75a166 None admin] PATH_INFO: `/v3/projects/73b78d25934e489f84431200ec41dd2d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.958068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61f38e-6c16-4487-8fc7-17478a75a166 None admin] RBAC: Authorizing `identity:get_project(project_id=73b78d25934e489f84431200ec41dd2d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:57.961216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61f38e-6c16-4487-8fc7-17478a75a166 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:57.963426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 68/135] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:34:57 2022] GET /identity/v3/projects/73b78d25934e489f84431200ec41dd2d => generated 312 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:57.971374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa2af0-7477-43a7-a26d-73ba4de820ab None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:57.972829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa2af0-7477-43a7-a26d-73ba4de820ab None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:57.973298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa2af0-7477-43a7-a26d-73ba4de820ab None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:57.976408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 69/136] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:57 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:57.977051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa2af0-7477-43a7-a26d-73ba4de820ab None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:57.992209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade1749-7fd5-409c-87e0-7d5253301816 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:58.029693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b42d1e-a4c8-46a4-bc12-badd307842ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:58.030523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b42d1e-a4c8-46a4-bc12-badd307842ec None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:58.030878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b42d1e-a4c8-46a4-bc12-badd307842ec None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:58.031253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b42d1e-a4c8-46a4-bc12-badd307842ec None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/users/3678a202c2fa4669a16786db2586e75d/roles/90097e8ce1394daba58b200de808ad59` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:58.033724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b42d1e-a4c8-46a4-bc12-badd307842ec None admin] RBAC: Authorizing `identity:create_grant(project_id=485f0f7bdc174746a34e4c85b269d3dd, user_id=3678a202c2fa4669a16786db2586e75d, role_id=90097e8ce1394daba58b200de808ad59)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:58.035299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b42d1e-a4c8-46a4-bc12-badd307842ec None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:58.052218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 68/137] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:34:57 2022] PUT /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/users/3678a202c2fa4669a16786db2586e75d/roles/90097e8ce1394daba58b200de808ad59 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:34:58.053680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c89842-8c32-4bc2-ae76-29268145d047 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:58.054077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c89842-8c32-4bc2-ae76-29268145d047 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:58.054424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c89842-8c32-4bc2-ae76-29268145d047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:58.075592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-72c89842-8c32-4bc2-ae76-29268145d047 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:58.077224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-72c89842-8c32-4bc2-ae76-29268145d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:58.134818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa2af0-7477-43a7-a26d-73ba4de820ab None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:58.135011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:58.136595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:58.165746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 69/138] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:34:58.193264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b51ec-b51a-496f-9a4c-3bfdbe66eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:58.194022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b51ec-b51a-496f-9a4c-3bfdbe66eca7 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:58.194304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b51ec-b51a-496f-9a4c-3bfdbe66eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:58.194580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b51ec-b51a-496f-9a4c-3bfdbe66eca7 None admin] PATH_INFO: `/v3/projects/73b78d25934e489f84431200ec41dd2d/users/3678a202c2fa4669a16786db2586e75d/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:58.199283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b51ec-b51a-496f-9a4c-3bfdbe66eca7 None admin] RBAC: Authorizing `identity:create_grant(project_id=73b78d25934e489f84431200ec41dd2d, user_id=3678a202c2fa4669a16786db2586e75d, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:58.200989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b51ec-b51a-496f-9a4c-3bfdbe66eca7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:58.221625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 70/139] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:34:57 2022] PUT /identity/v3/projects/73b78d25934e489f84431200ec41dd2d/users/3678a202c2fa4669a16786db2586e75d/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:34:58.535465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c89842-8c32-4bc2-ae76-29268145d047 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:58.652383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448843-2395-4fe1-8161-d9a60ae11d20 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:58.652917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448843-2395-4fe1-8161-d9a60ae11d20 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:58.653185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448843-2395-4fe1-8161-d9a60ae11d20 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:58.655345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 71/140] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:58.661428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0df2-c4cf-4a64-8429-3778429ec58d None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:58.661720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0df2-c4cf-4a64-8429-3778429ec58d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:58.662002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c0df2-c4cf-4a64-8429-3778429ec58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:58.663840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c89842-8c32-4bc2-ae76-29268145d047 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:58.703362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fb679-2d39-41cb-94da-a9a1b632ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:58.704067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb679-2d39-41cb-94da-a9a1b632ef02 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:58.704369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb679-2d39-41cb-94da-a9a1b632ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:58.704646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb679-2d39-41cb-94da-a9a1b632ef02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:58.705815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fb679-2d39-41cb-94da-a9a1b632ef02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:58.707289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fb679-2d39-41cb-94da-a9a1b632ef02 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:58.709333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 70/141] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:34:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:58.710328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-940c0df2-c4cf-4a64-8429-3778429ec58d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:58.711612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-940c0df2-c4cf-4a64-8429-3778429ec58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:58.723753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fb679-2d39-41cb-94da-a9a1b632ef02 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:58.847746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:58.863918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 72/142] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:58.937050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fb679-2d39-41cb-94da-a9a1b632ef02 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:58.992939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52990dd-d298-4b4c-a998-3f58bc8ce035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:58.993787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52990dd-d298-4b4c-a998-3f58bc8ce035 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:58.993942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52990dd-d298-4b4c-a998-3f58bc8ce035 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:58.994215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52990dd-d298-4b4c-a998-3f58bc8ce035 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:58.995100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52990dd-d298-4b4c-a998-3f58bc8ce035 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:58.997068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52990dd-d298-4b4c-a998-3f58bc8ce035 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:59.032332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 71/143] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:34:58 2022] POST /identity/v3/users => generated 283 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:34:59.309147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c0df2-c4cf-4a64-8429-3778429ec58d None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:59.422257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d029bf-62ee-4a00-b71f-529d6ee53a85 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:59.422620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d029bf-62ee-4a00-b71f-529d6ee53a85 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:59.422811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d029bf-62ee-4a00-b71f-529d6ee53a85 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:59.424190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 72/144] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:34:59.433635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314b408-e918-41e5-a4a1-c55047f46dc4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:59.433635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314b408-e918-41e5-a4a1-c55047f46dc4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:59.433635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314b408-e918-41e5-a4a1-c55047f46dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:59.447949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:59.456206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3314b408-e918-41e5-a4a1-c55047f46dc4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:34:59.458433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c0df2-c4cf-4a64-8429-3778429ec58d None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:59.461657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3314b408-e918-41e5-a4a1-c55047f46dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:34:59.536815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42bd6d-ecba-4f26-b58b-ca8cd098777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:59.538496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42bd6d-ecba-4f26-b58b-ca8cd098777e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:59.539494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42bd6d-ecba-4f26-b58b-ca8cd098777e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:59.540453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42bd6d-ecba-4f26-b58b-ca8cd098777e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:59.569521 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42bd6d-ecba-4f26-b58b-ca8cd098777e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-0a42bd6d-ecba-4f26-b58b-ca8cd098777e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:34:59.576836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:34:59.577968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:34:59.578834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:34:59.581559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 73/145] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:34:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:34:59.589590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42bd6d-ecba-4f26-b58b-ca8cd098777e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:59.637397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 73/146] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:34:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:34:59.749910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42bd6d-ecba-4f26-b58b-ca8cd098777e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:34:59.803261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:34:59.804280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:34:59.804762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:34:59.805313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:34:59.806280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] RBAC: Adding query filter params (name=reader) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:34:59.806822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:34:59.808509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:34:59.823205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 74/147] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:34:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:34:59.829589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314b408-e918-41e5-a4a1-c55047f46dc4 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:34:59.869731 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:34:59.954613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314b408-e918-41e5-a4a1-c55047f46dc4 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:00.007110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6703e-7f36-410d-a986-35dc9f7c4d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:00.009359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6703e-7f36-410d-a986-35dc9f7c4d1d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:00.024482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6703e-7f36-410d-a986-35dc9f7c4d1d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:00.025028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6703e-7f36-410d-a986-35dc9f7c4d1d None admin] PATH_INFO: `/v3/users/7c1dff78994d4e459ebcedaf94641bea` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:00.026812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6703e-7f36-410d-a986-35dc9f7c4d1d None admin] RBAC: Authorizing `identity:get_user(user_id=7c1dff78994d4e459ebcedaf94641bea)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:00.028593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6703e-7f36-410d-a986-35dc9f7c4d1d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:00.030598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 74/148] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:34:59 2022] GET /identity/v3/users/7c1dff78994d4e459ebcedaf94641bea => generated 289 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:00.038352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:00.062281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:00.171530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc88b83-aa56-477a-925b-600406d8c15f None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:00.231530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc001c8-0b10-484e-a95d-783cffe8f8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:00.232620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc001c8-0b10-484e-a95d-783cffe8f8f8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:00.233118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc001c8-0b10-484e-a95d-783cffe8f8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:00.233630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc001c8-0b10-484e-a95d-783cffe8f8f8 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:00.235756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc001c8-0b10-484e-a95d-783cffe8f8f8 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:00.238700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc001c8-0b10-484e-a95d-783cffe8f8f8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:00.241387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 75/149] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:00 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:00.247103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6703e-7f36-410d-a986-35dc9f7c4d1d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:00.381236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6703e-7f36-410d-a986-35dc9f7c4d1d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:00.386683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:00.425159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc001c8-0b10-484e-a95d-783cffe8f8f8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:00.436654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094ff59-b030-4d6a-a2eb-0823a95c507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:00.437329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094ff59-b030-4d6a-a2eb-0823a95c507a None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:00.437628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094ff59-b030-4d6a-a2eb-0823a95c507a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:00.437913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094ff59-b030-4d6a-a2eb-0823a95c507a None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/users/7c1dff78994d4e459ebcedaf94641bea/roles/935445ed062541ceb33d5021be2b092a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:00.444597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094ff59-b030-4d6a-a2eb-0823a95c507a None admin] RBAC: Authorizing `identity:create_grant(project_id=485f0f7bdc174746a34e4c85b269d3dd, user_id=7c1dff78994d4e459ebcedaf94641bea, role_id=935445ed062541ceb33d5021be2b092a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:00.446417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c094ff59-b030-4d6a-a2eb-0823a95c507a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:00.470521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 75/150] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:35:00 2022] PUT /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/users/7c1dff78994d4e459ebcedaf94641bea/roles/935445ed062541ceb33d5021be2b092a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:00.566404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:00.589319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc001c8-0b10-484e-a95d-783cffe8f8f8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:00.636579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd1046-d635-4107-a1db-0691720a1192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:00.637242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd1046-d635-4107-a1db-0691720a1192 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:00.637520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd1046-d635-4107-a1db-0691720a1192 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:00.637850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd1046-d635-4107-a1db-0691720a1192 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:00.651581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bd1046-d635-4107-a1db-0691720a1192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-74bd1046-d635-4107-a1db-0691720a1192 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:00.657229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:00.658926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:00.660420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 76/151] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:00.665053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094ff59-b030-4d6a-a2eb-0823a95c507a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:00.799946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094ff59-b030-4d6a-a2eb-0823a95c507a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:00.840288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:00.840917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:00.841854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:00.842372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:00.843071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:00.843378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:00.844791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:00.854342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 76/152] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:00.860094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd1046-d635-4107-a1db-0691720a1192 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:00.983980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd1046-d635-4107-a1db-0691720a1192 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:01.041980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f346-0231-44d8-b710-238757be23fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:01.042639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f346-0231-44d8-b710-238757be23fb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.042918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f346-0231-44d8-b710-238757be23fb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.043193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8f346-0231-44d8-b710-238757be23fb None admin] PATH_INFO: `/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.044681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8f346-0231-44d8-b710-238757be23fb None admin] RBAC: Authorizing `identity:get_user(user_id=6a5ebebe92dd4d8b9d923b7a40d319f7)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:01.046298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c8f346-0231-44d8-b710-238757be23fb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:01.048061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 77/153] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:00 2022] GET /identity/v3/users/6a5ebebe92dd4d8b9d923b7a40d319f7 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:01.053952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:01.087480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:01.174331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76fdbb-0b9b-412b-8f8d-941208c5af59 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:01.211929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19842cd1-925d-43e9-b80e-6f2eb52650d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:01.212559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19842cd1-925d-43e9-b80e-6f2eb52650d1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.212843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19842cd1-925d-43e9-b80e-6f2eb52650d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.213117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19842cd1-925d-43e9-b80e-6f2eb52650d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.288939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19842cd1-925d-43e9-b80e-6f2eb52650d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:01.290468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19842cd1-925d-43e9-b80e-6f2eb52650d1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:01.320270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:01.402357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 77/154] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:01.408433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f346-0231-44d8-b710-238757be23fb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:01.424337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:01.536415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8f346-0231-44d8-b710-238757be23fb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:01.586702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41ab3f-8151-4ed4-805e-8819a4931010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:01.587278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ab3f-8151-4ed4-805e-8819a4931010 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.587558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ab3f-8151-4ed4-805e-8819a4931010 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.587827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ab3f-8151-4ed4-805e-8819a4931010 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.589052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41ab3f-8151-4ed4-805e-8819a4931010 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:01.590573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41ab3f-8151-4ed4-805e-8819a4931010 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:01.593301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 78/155] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:01 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:01.599260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19842cd1-925d-43e9-b80e-6f2eb52650d1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:01.690943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057a601-b8c4-42c6-9956-a3df51e97351 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.690943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057a601-b8c4-42c6-9956-a3df51e97351 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.690943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6057a601-b8c4-42c6-9956-a3df51e97351 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.690943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 79/156] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:01.698371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0feb-abcb-4614-b76b-14149d526921 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.698371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0feb-abcb-4614-b76b-14149d526921 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.698371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e0feb-abcb-4614-b76b-14149d526921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.729308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19842cd1-925d-43e9-b80e-6f2eb52650d1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:01.735539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3d5e0feb-abcb-4614-b76b-14149d526921 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:01.737170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3d5e0feb-abcb-4614-b76b-14149d526921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:01.762024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:01.768877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14a69a-4a92-4fd5-90c8-1635132e25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:01.769930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a69a-4a92-4fd5-90c8-1635132e25f3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.770372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a69a-4a92-4fd5-90c8-1635132e25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.770796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a69a-4a92-4fd5-90c8-1635132e25f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.771693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14a69a-4a92-4fd5-90c8-1635132e25f3 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd, user.id=6a5ebebe92dd4d8b9d923b7a40d319f7) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:01.772439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14a69a-4a92-4fd5-90c8-1635132e25f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:01.774802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14a69a-4a92-4fd5-90c8-1635132e25f3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:01.782658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 78/157] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:01 2022] GET /identity/v3/role_assignments?user.id=6a5ebebe92dd4d8b9d923b7a40d319f7&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=b698800ceec3460a8519ea627650da0f => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:01.784236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c69e0d-9ab8-4ea2-b4d6-803d92e821c2 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.784886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c69e0d-9ab8-4ea2-b4d6-803d92e821c2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.785455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c69e0d-9ab8-4ea2-b4d6-803d92e821c2 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.791703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 79/158] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:01.798663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17d2c-f17e-4a74-9b4a-a333dac638d7 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.799328 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17d2c-f17e-4a74-9b4a-a333dac638d7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.799762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a17d2c-f17e-4a74-9b4a-a333dac638d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.827987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e4a17d2c-f17e-4a74-9b4a-a333dac638d7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:01.829817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e4a17d2c-f17e-4a74-9b4a-a333dac638d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:01.954831 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 80/159] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:01.956259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cb5ea-b0ef-4a77-b9ca-0f2407b3add0 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.956542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cb5ea-b0ef-4a77-b9ca-0f2407b3add0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.956817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cb5ea-b0ef-4a77-b9ca-0f2407b3add0 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:01.958192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 81/160] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:01.968135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 80/161] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:01.968320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd01f2c-99de-4f04-83b4-8d0eef58829e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:01.968536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd01f2c-99de-4f04-83b4-8d0eef58829e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:01.968728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd01f2c-99de-4f04-83b4-8d0eef58829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:02.012635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-acd01f2c-99de-4f04-83b4-8d0eef58829e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:02.012635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-acd01f2c-99de-4f04-83b4-8d0eef58829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:02.030071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:02.123858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 82/162] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 19:35:02.587437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17d2c-f17e-4a74-9b4a-a333dac638d7 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:02.603146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd01f2c-99de-4f04-83b4-8d0eef58829e None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:02.858335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a17d2c-f17e-4a74-9b4a-a333dac638d7 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:02.870598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd01f2c-99de-4f04-83b4-8d0eef58829e None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:02.914894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:02.954133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e174b-a123-47b9-bcbc-4d23a41d41e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:02.954788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e174b-a123-47b9-bcbc-4d23a41d41e9 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:02.955071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e174b-a123-47b9-bcbc-4d23a41d41e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:02.955342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e174b-a123-47b9-bcbc-4d23a41d41e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:02.958341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480086a-10bc-4e02-bf45-227292333396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:02.958994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480086a-10bc-4e02-bf45-227292333396 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:02.959270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480086a-10bc-4e02-bf45-227292333396 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:02.959540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480086a-10bc-4e02-bf45-227292333396 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:02.979837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:02.982762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e174b-a123-47b9-bcbc-4d23a41d41e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-933e174b-a123-47b9-bcbc-4d23a41d41e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:02.987956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:02.988982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:02.990071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 81/163] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:02.992214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480086a-10bc-4e02-bf45-227292333396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:02.994786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e174b-a123-47b9-bcbc-4d23a41d41e9 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-a480086a-10bc-4e02-bf45-227292333396 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:03.001489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:03.002485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:03.003275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:03.006896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 83/164] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:03.007735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480086a-10bc-4e02-bf45-227292333396 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:03.251610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e174b-a123-47b9-bcbc-4d23a41d41e9 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:03.294267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480086a-10bc-4e02-bf45-227292333396 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:03.328177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c13aa-cdab-47af-9a35-6954fc5718ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:03.329199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c13aa-cdab-47af-9a35-6954fc5718ff None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:03.329700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c13aa-cdab-47af-9a35-6954fc5718ff None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:03.330162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c13aa-cdab-47af-9a35-6954fc5718ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:03.357405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c13aa-cdab-47af-9a35-6954fc5718ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-f46c13aa-cdab-47af-9a35-6954fc5718ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:03.362428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:03.363337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:03.364144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:03.368023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 82/165] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:03.368914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c13aa-cdab-47af-9a35-6954fc5718ff None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:03.402325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:03.402958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:03.403229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:03.403519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:03.404327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:03.404630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:03.406070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:03.429112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 84/166] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:03.430013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:03.452790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:03.538244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:03.574367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c13aa-cdab-47af-9a35-6954fc5718ff None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:03.639081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc0bad-ef06-49fa-b4e2-c7547adfa2b4 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:03.644208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17775168-0200-4769-9565-31aa86039a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:03.645319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17775168-0200-4769-9565-31aa86039a12 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:03.645843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17775168-0200-4769-9565-31aa86039a12 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:03.646395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17775168-0200-4769-9565-31aa86039a12 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:03.647637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17775168-0200-4769-9565-31aa86039a12 None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:03.648186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17775168-0200-4769-9565-31aa86039a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:03.649785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17775168-0200-4769-9565-31aa86039a12 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:03.673546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 83/167] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:03.674441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17775168-0200-4769-9565-31aa86039a12 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:03.726308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:03.728027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:03.728303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:03.728577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:03.729253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:03.729840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:03.732671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:03.756932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 85/168] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:03.757830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:03.882848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17775168-0200-4769-9565-31aa86039a12 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:03.885895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:03.923606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2598d9-9e3b-4c42-b016-a69821c29dd2 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:03.950082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:03.964823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98558c-9b42-40e8-8f6f-81f5b260764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:03.966849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98558c-9b42-40e8-8f6f-81f5b260764b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:03.967351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98558c-9b42-40e8-8f6f-81f5b260764b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:03.967851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98558c-9b42-40e8-8f6f-81f5b260764b None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:03.969255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98558c-9b42-40e8-8f6f-81f5b260764b None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:03.970956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98558c-9b42-40e8-8f6f-81f5b260764b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:03.973175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 86/169] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:03 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:03.973795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019f01-7867-45d8-94df-7c762137c2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:03.974459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019f01-7867-45d8-94df-7c762137c2fb None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:03.974676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98558c-9b42-40e8-8f6f-81f5b260764b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:03.974850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019f01-7867-45d8-94df-7c762137c2fb None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:03.975130 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9019f01-7867-45d8-94df-7c762137c2fb None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:03.980679 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9019f01-7867-45d8-94df-7c762137c2fb None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:03.982309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9019f01-7867-45d8-94df-7c762137c2fb None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:03.984884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 84/170] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:03 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:03.985122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019f01-7867-45d8-94df-7c762137c2fb None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:04.121716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98558c-9b42-40e8-8f6f-81f5b260764b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:04.173184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d42ebc-b64b-40dd-945a-2f1434c4aca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:04.174133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d42ebc-b64b-40dd-945a-2f1434c4aca4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.174420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d42ebc-b64b-40dd-945a-2f1434c4aca4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.174696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d42ebc-b64b-40dd-945a-2f1434c4aca4 None admin] PATH_INFO: `/v3/users/3678a202c2fa4669a16786db2586e75d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.178525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d42ebc-b64b-40dd-945a-2f1434c4aca4 None admin] RBAC: Authorizing `identity:get_user(user_id=3678a202c2fa4669a16786db2586e75d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:04.180092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d42ebc-b64b-40dd-945a-2f1434c4aca4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:04.189648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 87/171] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:03 2022] GET /identity/v3/users/3678a202c2fa4669a16786db2586e75d => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:04.189648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d42ebc-b64b-40dd-945a-2f1434c4aca4 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:04.212569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9019f01-7867-45d8-94df-7c762137c2fb None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:04.248561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:04.311243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d42ebc-b64b-40dd-945a-2f1434c4aca4 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:04.314418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68e96b-5e39-46e7-b53d-bb8d62cb0748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:04.315058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68e96b-5e39-46e7-b53d-bb8d62cb0748 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.315342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68e96b-5e39-46e7-b53d-bb8d62cb0748 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.315622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68e96b-5e39-46e7-b53d-bb8d62cb0748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.353494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b377c4-2c1e-4c3a-92b1-5d2198685b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:04.354156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b377c4-2c1e-4c3a-92b1-5d2198685b23 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.354436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b377c4-2c1e-4c3a-92b1-5d2198685b23 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.354710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b377c4-2c1e-4c3a-92b1-5d2198685b23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.361804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:04.403685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68e96b-5e39-46e7-b53d-bb8d62cb0748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:04.405511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68e96b-5e39-46e7-b53d-bb8d62cb0748 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:04.455499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b377c4-2c1e-4c3a-92b1-5d2198685b23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:04.457654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b377c4-2c1e-4c3a-92b1-5d2198685b23 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:04.523978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:04.543333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 85/172] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:04.547085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97bc8-456e-4be2-8e80-008cb4211cb2 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.547843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97bc8-456e-4be2-8e80-008cb4211cb2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.548828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b97bc8-456e-4be2-8e80-008cb4211cb2 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.550416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 86/173] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:04.554513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b97bc8-456e-4be2-8e80-008cb4211cb2 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:04.586054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 88/174] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:04.587188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ae1ad-bc2f-429e-a200-00c74e41f545 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.587474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ae1ad-bc2f-429e-a200-00c74e41f545 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.587747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ae1ad-bc2f-429e-a200-00c74e41f545 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.589172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 89/175] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:04.594295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257ea9-51fb-48d0-a400-c63ed0eac266 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.594575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257ea9-51fb-48d0-a400-c63ed0eac266 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.595184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257ea9-51fb-48d0-a400-c63ed0eac266 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.597113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 90/176] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:04.601755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257ea9-51fb-48d0-a400-c63ed0eac266 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:04.650299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:04.687389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b97bc8-456e-4be2-8e80-008cb4211cb2 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:04.731517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c271a-c32a-499f-87aa-655c7563c9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:04.732339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c271a-c32a-499f-87aa-655c7563c9a2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.732634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c271a-c32a-499f-87aa-655c7563c9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.732907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c271a-c32a-499f-87aa-655c7563c9a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.734638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257ea9-51fb-48d0-a400-c63ed0eac266 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:04.776667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b9166-9754-4562-8ee1-69691e4391cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:04.777315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9166-9754-4562-8ee1-69691e4391cc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.777594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9166-9754-4562-8ee1-69691e4391cc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.777895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b9166-9754-4562-8ee1-69691e4391cc None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.779071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b9166-9754-4562-8ee1-69691e4391cc None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:04.781072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b9166-9754-4562-8ee1-69691e4391cc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:04.788562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 91/177] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:04 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:04.788562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd97593-a44d-4b60-907a-3c2783679565 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.788562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd97593-a44d-4b60-907a-3c2783679565 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.788562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd97593-a44d-4b60-907a-3c2783679565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.814811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-2cd97593-a44d-4b60-907a-3c2783679565 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:04.816640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2cd97593-a44d-4b60-907a-3c2783679565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:04.821665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c271a-c32a-499f-87aa-655c7563c9a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:04.823516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c271a-c32a-499f-87aa-655c7563c9a2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:04.853426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:04.907415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:04.931855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 92/178] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:04.933459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd97593-a44d-4b60-907a-3c2783679565 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:04.959786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 87/179] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:04.963181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d5168-56e1-4a08-8759-7ff5f1ef93fb None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:04.963475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d5168-56e1-4a08-8759-7ff5f1ef93fb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:04.963756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d5168-56e1-4a08-8759-7ff5f1ef93fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:04.989916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8e7d5168-56e1-4a08-8759-7ff5f1ef93fb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:04.991033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8e7d5168-56e1-4a08-8759-7ff5f1ef93fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:05.076429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd97593-a44d-4b60-907a-3c2783679565 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:05.078931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 88/180] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:05.080365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2d708-b64d-4da0-a9e6-004d99690c47 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.080665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2d708-b64d-4da0-a9e6-004d99690c47 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.080942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2d708-b64d-4da0-a9e6-004d99690c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.104221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e1c2d708-b64d-4da0-a9e6-004d99690c47 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:05.105327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e1c2d708-b64d-4da0-a9e6-004d99690c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:05.125143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b33a44-d37c-49c4-81d2-95ded9a4da25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:05.126185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33a44-d37c-49c4-81d2-95ded9a4da25 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.126712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33a44-d37c-49c4-81d2-95ded9a4da25 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.127198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b33a44-d37c-49c4-81d2-95ded9a4da25 None admin] PATH_INFO: `/v3/projects/73b78d25934e489f84431200ec41dd2d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.128665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b33a44-d37c-49c4-81d2-95ded9a4da25 None admin] RBAC: Authorizing `identity:get_project(project_id=73b78d25934e489f84431200ec41dd2d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:05.130528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b33a44-d37c-49c4-81d2-95ded9a4da25 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:05.132743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 93/181] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:04 2022] GET /identity/v3/projects/73b78d25934e489f84431200ec41dd2d => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:05.134078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b33a44-d37c-49c4-81d2-95ded9a4da25 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:05.166217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:05.187626 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 89/182] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:05.189239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2d708-b64d-4da0-a9e6-004d99690c47 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:05.240038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:05.282088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b33a44-d37c-49c4-81d2-95ded9a4da25 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:05.327105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861920ba-c0cd-41ee-a72c-332771c95523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:05.328308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861920ba-c0cd-41ee-a72c-332771c95523 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.328921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861920ba-c0cd-41ee-a72c-332771c95523 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.329304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861920ba-c0cd-41ee-a72c-332771c95523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.330296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861920ba-c0cd-41ee-a72c-332771c95523 None admin] RBAC: Adding query filter params (role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd, user.id=3678a202c2fa4669a16786db2586e75d) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:05.330680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861920ba-c0cd-41ee-a72c-332771c95523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:05.332214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861920ba-c0cd-41ee-a72c-332771c95523 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:05.340481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 94/183] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:05 2022] GET /identity/v3/role_assignments?user.id=3678a202c2fa4669a16786db2586e75d&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:05.343593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2d708-b64d-4da0-a9e6-004d99690c47 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:05.352648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3817e-6a20-4f35-80ef-52683bed4458 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.353418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3817e-6a20-4f35-80ef-52683bed4458 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.353900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3817e-6a20-4f35-80ef-52683bed4458 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.355596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 95/184] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:05 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:05.363786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3817e-6a20-4f35-80ef-52683bed4458 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:05.397863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa7470-050a-4d6b-9e8e-1fb85fe1a57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:05.398712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa7470-050a-4d6b-9e8e-1fb85fe1a57e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.398915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa7470-050a-4d6b-9e8e-1fb85fe1a57e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.399162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa7470-050a-4d6b-9e8e-1fb85fe1a57e None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.400369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa7470-050a-4d6b-9e8e-1fb85fe1a57e None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:05.402031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa7470-050a-4d6b-9e8e-1fb85fe1a57e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:05.404457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 90/185] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:05 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:05.405298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa7470-050a-4d6b-9e8e-1fb85fe1a57e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:05.505861 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:05.524500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3817e-6a20-4f35-80ef-52683bed4458 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:05.549663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:05.554349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa7470-050a-4d6b-9e8e-1fb85fe1a57e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:05.574937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:05.576466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.576920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.577374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.578669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] RBAC: Adding query filter params (role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=73b78d25934e489f84431200ec41dd2d, user.id=3678a202c2fa4669a16786db2586e75d) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:05.579175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:05.581545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:05.592254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 96/186] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:05 2022] GET /identity/v3/role_assignments?user.id=3678a202c2fa4669a16786db2586e75d&scope.project.id=73b78d25934e489f84431200ec41dd2d&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:05.593859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:05.606230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c01a03-abb3-4607-b9dc-14ddf571b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:05.607187 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c01a03-abb3-4607-b9dc-14ddf571b402 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.607623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c01a03-abb3-4607-b9dc-14ddf571b402 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.608072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c01a03-abb3-4607-b9dc-14ddf571b402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c01a03-abb3-4607-b9dc-14ddf571b402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-d3c01a03-abb3-4607-b9dc-14ddf571b402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:05.626351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:05.627309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:05.628124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 91/187] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:05.629981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d920ce-2bb7-4418-8703-db6890a3f4e2 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.629981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d920ce-2bb7-4418-8703-db6890a3f4e2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.629981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d920ce-2bb7-4418-8703-db6890a3f4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.650981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e8d920ce-2bb7-4418-8703-db6890a3f4e2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:05.652339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e8d920ce-2bb7-4418-8703-db6890a3f4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:05.731516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea2d72-249e-493d-b133-79f143016d4a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:05.761804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 92/188] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:05.761804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d920ce-2bb7-4418-8703-db6890a3f4e2 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:05.799344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6fd5c-6aaa-4bf8-98cb-96734abb0b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:05.800592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6fd5c-6aaa-4bf8-98cb-96734abb0b26 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.800879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6fd5c-6aaa-4bf8-98cb-96734abb0b26 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.801158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6fd5c-6aaa-4bf8-98cb-96734abb0b26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.813559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6fd5c-6aaa-4bf8-98cb-96734abb0b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-bed6fd5c-6aaa-4bf8-98cb-96734abb0b26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:05.820398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:05.822054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 97/189] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:05.823773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6fd5c-6aaa-4bf8-98cb-96734abb0b26 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:05.840386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:05.903102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:05.921957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d920ce-2bb7-4418-8703-db6890a3f4e2 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:05.960982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6fd5c-6aaa-4bf8-98cb-96734abb0b26 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:05.977265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:05.978556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:05.978987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:05.979442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:05.981237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] RBAC: Adding query filter params (role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd, user.id=3678a202c2fa4669a16786db2586e75d) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:05.981941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:05.985321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.005641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 93/190] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:05 2022] GET /identity/v3/role_assignments?user.id=3678a202c2fa4669a16786db2586e75d&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=90097e8ce1394daba58b200de808ad59 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:06.005641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.008972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418b887-654e-41a2-a39f-9a038875a06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.009760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b887-654e-41a2-a39f-9a038875a06c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.012731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b887-654e-41a2-a39f-9a038875a06c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.012731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b887-654e-41a2-a39f-9a038875a06c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.023550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418b887-654e-41a2-a39f-9a038875a06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-f418b887-654e-41a2-a39f-9a038875a06c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:06.029463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:06.031346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 98/191] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:06.032826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418b887-654e-41a2-a39f-9a038875a06c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.137883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:06.170356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e3f45-7793-4d5d-a1f5-cd7240e02dbd None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.177019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418b887-654e-41a2-a39f-9a038875a06c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.225338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.226440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.226977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.227505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.228684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:06.229164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:06.231364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.232596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.232730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.232876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.233154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.233894 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:06.234200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:06.235901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.246097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 99/192] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:06.246963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.247668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 94/193] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:06.249030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.271422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:06.369323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a68f1-311f-49f6-84ec-d79b25823a66 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.388864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe37938-002c-4b58-bec1-44c22a63242b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.408468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.409303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.409720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.410138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.411150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:06.411538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:06.412936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.414614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:06.424440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 100/194] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:06.424440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.437508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492bb9c-2479-4a08-b80f-f5f8762cd4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.438188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492bb9c-2479-4a08-b80f-f5f8762cd4b3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.438469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492bb9c-2479-4a08-b80f-f5f8762cd4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.438739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492bb9c-2479-4a08-b80f-f5f8762cd4b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.439868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492bb9c-2479-4a08-b80f-f5f8762cd4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:06.441339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492bb9c-2479-4a08-b80f-f5f8762cd4b3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.442983 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 95/195] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:06.443764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492bb9c-2479-4a08-b80f-f5f8762cd4b3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.535945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8718bb-571d-4160-b2b1-4fdd786f98d1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.536247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:06.560997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492bb9c-2479-4a08-b80f-f5f8762cd4b3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.576431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc138cc2-e713-46e3-8409-550425d303d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.577125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc138cc2-e713-46e3-8409-550425d303d0 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.577400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc138cc2-e713-46e3-8409-550425d303d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.577700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc138cc2-e713-46e3-8409-550425d303d0 None admin] PATH_INFO: `/v3/users/733438bf54304476bda847bd56fd5b81` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.592644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc138cc2-e713-46e3-8409-550425d303d0 None admin] RBAC: Authorizing `identity:get_user(user_id=733438bf54304476bda847bd56fd5b81)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:06.594158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc138cc2-e713-46e3-8409-550425d303d0 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.595843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 101/196] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:06 2022] GET /identity/v3/users/733438bf54304476bda847bd56fd5b81 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:06.596622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc138cc2-e713-46e3-8409-550425d303d0 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.601339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761f15c-2063-4d7e-9a15-232d2af08d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.602023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761f15c-2063-4d7e-9a15-232d2af08d0d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.602309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761f15c-2063-4d7e-9a15-232d2af08d0d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.602588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761f15c-2063-4d7e-9a15-232d2af08d0d None admin] PATH_INFO: `/v3/users/733438bf54304476bda847bd56fd5b81` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.603790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761f15c-2063-4d7e-9a15-232d2af08d0d None admin] RBAC: Authorizing `identity:get_user(user_id=733438bf54304476bda847bd56fd5b81)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:06.605255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761f15c-2063-4d7e-9a15-232d2af08d0d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.606941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 96/197] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:06 2022] GET /identity/v3/users/733438bf54304476bda847bd56fd5b81 => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:06.607705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761f15c-2063-4d7e-9a15-232d2af08d0d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.658620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:06.712645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc138cc2-e713-46e3-8409-550425d303d0 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.727395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761f15c-2063-4d7e-9a15-232d2af08d0d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.756430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e814a0-770d-4629-8d84-13e1bc5347cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.757088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e814a0-770d-4629-8d84-13e1bc5347cb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.757359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e814a0-770d-4629-8d84-13e1bc5347cb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.757647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e814a0-770d-4629-8d84-13e1bc5347cb None admin] PATH_INFO: `/v3/users/7c1dff78994d4e459ebcedaf94641bea` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.758806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e814a0-770d-4629-8d84-13e1bc5347cb None admin] RBAC: Authorizing `identity:get_user(user_id=7c1dff78994d4e459ebcedaf94641bea)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:06.760273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e814a0-770d-4629-8d84-13e1bc5347cb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.761916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 102/198] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:06 2022] GET /identity/v3/users/7c1dff78994d4e459ebcedaf94641bea => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:06.762886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e814a0-770d-4629-8d84-13e1bc5347cb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.774776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34117e-49e5-4b00-9612-db6c7715eb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.775397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34117e-49e5-4b00-9612-db6c7715eb8e None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.775677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34117e-49e5-4b00-9612-db6c7715eb8e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.775945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34117e-49e5-4b00-9612-db6c7715eb8e None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:06.776608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34117e-49e5-4b00-9612-db6c7715eb8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:06.778102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34117e-49e5-4b00-9612-db6c7715eb8e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:06.800023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 97/199] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:06 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:06.800779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34117e-49e5-4b00-9612-db6c7715eb8e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:06.810886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:06.903739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e814a0-770d-4629-8d84-13e1bc5347cb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.932603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:06.944800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34117e-49e5-4b00-9612-db6c7715eb8e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:06.952423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9748e4c-c3cb-4321-bfb0-9e3019148d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:06.953135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9748e4c-c3cb-4321-bfb0-9e3019148d44 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:06.953422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9748e4c-c3cb-4321-bfb0-9e3019148d44 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:06.953717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9748e4c-c3cb-4321-bfb0-9e3019148d44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:07.006065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e8be9-54f3-4da9-b451-eb67f42f16e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:07.006442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e8be9-54f3-4da9-b451-eb67f42f16e2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:07.006724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e8be9-54f3-4da9-b451-eb67f42f16e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:07.007668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e8be9-54f3-4da9-b451-eb67f42f16e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:07.045000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9748e4c-c3cb-4321-bfb0-9e3019148d44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:07.046549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9748e4c-c3cb-4321-bfb0-9e3019148d44 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:07.116723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e8be9-54f3-4da9-b451-eb67f42f16e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:07.118307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e8be9-54f3-4da9-b451-eb67f42f16e2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:07.131752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:07.180052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 103/200] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:07.182664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9748e4c-c3cb-4321-bfb0-9e3019148d44 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:07.211716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:07.257379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 98/201] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:07.259220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e8be9-54f3-4da9-b451-eb67f42f16e2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:07.314203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9748e4c-c3cb-4321-bfb0-9e3019148d44 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:07.362200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebfb9d-ffb0-4543-b95d-a83b8dd25149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:07.363111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebfb9d-ffb0-4543-b95d-a83b8dd25149 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:07.363539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebfb9d-ffb0-4543-b95d-a83b8dd25149 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:07.363984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebfb9d-ffb0-4543-b95d-a83b8dd25149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:07.382785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e8be9-54f3-4da9-b451-eb67f42f16e2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:07.393130 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:07.424707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857bae6-b224-4a90-9b71-918bd35aa0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:07.425665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857bae6-b224-4a90-9b71-918bd35aa0e9 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:07.426119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857bae6-b224-4a90-9b71-918bd35aa0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:07.426540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857bae6-b224-4a90-9b71-918bd35aa0e9 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:07.427936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857bae6-b224-4a90-9b71-918bd35aa0e9 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:07.429637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6857bae6-b224-4a90-9b71-918bd35aa0e9 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:07.431654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 99/202] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:07 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:07.432293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857bae6-b224-4a90-9b71-918bd35aa0e9 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:07.459872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebfb9d-ffb0-4543-b95d-a83b8dd25149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:07.461375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ebfb9d-ffb0-4543-b95d-a83b8dd25149 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:07.489724 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:07.591272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857bae6-b224-4a90-9b71-918bd35aa0e9 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:07.609141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 104/203] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:07.611307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebfb9d-ffb0-4543-b95d-a83b8dd25149 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:07.632136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1c7cd-fe24-4ddc-9d5b-8d74eeb25ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:07.632778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c7cd-fe24-4ddc-9d5b-8d74eeb25ccd None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:07.633119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c7cd-fe24-4ddc-9d5b-8d74eeb25ccd None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:07.633328 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1c7cd-fe24-4ddc-9d5b-8d74eeb25ccd None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:07.634613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1c7cd-fe24-4ddc-9d5b-8d74eeb25ccd None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:07.636204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1c7cd-fe24-4ddc-9d5b-8d74eeb25ccd None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:07.638121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 100/204] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:07 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:07.643637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1c7cd-fe24-4ddc-9d5b-8d74eeb25ccd None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:07.698020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:07.756199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebfb9d-ffb0-4543-b95d-a83b8dd25149 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:07.770859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1c7cd-fe24-4ddc-9d5b-8d74eeb25ccd None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:07.789799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:07.812045 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:07.813599 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:07.814270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:07.814962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:07.815965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc, user.id=733438bf54304476bda847bd56fd5b81) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:07.816467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5252-dc59-439f-ab30-f499f8d71057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:07.816720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:07.817330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5252-dc59-439f-ab30-f499f8d71057 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:07.817720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5252-dc59-439f-ab30-f499f8d71057 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:07.818067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5252-dc59-439f-ab30-f499f8d71057 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:07.818950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:07.819353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f5252-dc59-439f-ab30-f499f8d71057 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:07.820981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f5252-dc59-439f-ab30-f499f8d71057 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:07.823091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 101/205] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:07 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:07.823941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5252-dc59-439f-ab30-f499f8d71057 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:07.836230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 105/206] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:07 2022] GET /identity/v3/role_assignments?user.id=733438bf54304476bda847bd56fd5b81&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=b698800ceec3460a8519ea627650da0f => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:07.837263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:07.976770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5252-dc59-439f-ab30-f499f8d71057 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:07.990056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:08.054975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27deb101-c70c-4cdc-8c18-7a24a2c30c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:08.055614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deb101-c70c-4cdc-8c18-7a24a2c30c44 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:08.055893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deb101-c70c-4cdc-8c18-7a24a2c30c44 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:08.056165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deb101-c70c-4cdc-8c18-7a24a2c30c44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:08.056972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27deb101-c70c-4cdc-8c18-7a24a2c30c44 None admin] RBAC: Adding query filter params (role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc, user.id=733438bf54304476bda847bd56fd5b81) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:08.057287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27deb101-c70c-4cdc-8c18-7a24a2c30c44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:08.058924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27deb101-c70c-4cdc-8c18-7a24a2c30c44 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:08.070802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 102/207] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:07 2022] GET /identity/v3/role_assignments?user.id=733438bf54304476bda847bd56fd5b81&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=90097e8ce1394daba58b200de808ad59 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:08.103337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63495075-beb2-4f6a-b54c-11d7466eff56 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:08.143141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acaec9-8a6f-48cb-b700-11b73d77b394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:08.143756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acaec9-8a6f-48cb-b700-11b73d77b394 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:08.144030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acaec9-8a6f-48cb-b700-11b73d77b394 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:08.144294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acaec9-8a6f-48cb-b700-11b73d77b394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:08.145027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acaec9-8a6f-48cb-b700-11b73d77b394 None admin] RBAC: Adding query filter params (role.id=935445ed062541ceb33d5021be2b092a, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd, user.id=7c1dff78994d4e459ebcedaf94641bea) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:08.145326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acaec9-8a6f-48cb-b700-11b73d77b394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:08.146811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acaec9-8a6f-48cb-b700-11b73d77b394 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:08.153621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 106/208] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:07 2022] GET /identity/v3/role_assignments?user.id=7c1dff78994d4e459ebcedaf94641bea&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=935445ed062541ceb33d5021be2b092a => generated 608 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:09.495375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba7f0-365a-49ac-a61a-3ae39ebb5176 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:09.495958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba7f0-365a-49ac-a61a-3ae39ebb5176 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:09.495958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba7f0-365a-49ac-a61a-3ae39ebb5176 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:09.497490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 103/209] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:09.503312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a76fd-ac1a-4b12-8b60-abce35947ee9 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:09.503815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a76fd-ac1a-4b12-8b60-abce35947ee9 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:09.503888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a76fd-ac1a-4b12-8b60-abce35947ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:09.524952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5c6a76fd-ac1a-4b12-8b60-abce35947ee9 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:09.526205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5c6a76fd-ac1a-4b12-8b60-abce35947ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:09.581484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:09.613561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 107/210] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:09.792807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35575e8-89e8-4f7f-ba43-f3cc319c290c None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:09.793103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35575e8-89e8-4f7f-ba43-f3cc319c290c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:09.793266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35575e8-89e8-4f7f-ba43-f3cc319c290c None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:09.795060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 104/211] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:09.800699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776db0f3-5ad6-42df-890c-7020a41f34f6 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:09.800981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776db0f3-5ad6-42df-890c-7020a41f34f6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:09.801256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776db0f3-5ad6-42df-890c-7020a41f34f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:09.820666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-776db0f3-5ad6-42df-890c-7020a41f34f6 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:09.821907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-776db0f3-5ad6-42df-890c-7020a41f34f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:09.899010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 108/212] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:09.902740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35575e8-89e8-4f7f-ba43-f3cc319c290c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:10.052617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35575e8-89e8-4f7f-ba43-f3cc319c290c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:10.094258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c255c19-952d-483a-967d-18dc99d36497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:10.095009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c255c19-952d-483a-967d-18dc99d36497 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.095298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c255c19-952d-483a-967d-18dc99d36497 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.095572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c255c19-952d-483a-967d-18dc99d36497 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.107585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c255c19-952d-483a-967d-18dc99d36497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-9c255c19-952d-483a-967d-18dc99d36497 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:10.113333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:10.114308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:10.115099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 105/213] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:10.120833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776db0f3-5ad6-42df-890c-7020a41f34f6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:10.239220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c255c19-952d-483a-967d-18dc99d36497 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:10.247836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776db0f3-5ad6-42df-890c-7020a41f34f6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:10.260402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:10.285659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:10.291009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:10.291645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.291922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.292191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.292872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:10.293174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:10.295155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:10.306374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 109/214] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:10.312738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:10.380890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c255c19-952d-483a-967d-18dc99d36497 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:10.426309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5ff4a-07ac-4464-974a-8d9360519d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:10.427255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5ff4a-07ac-4464-974a-8d9360519d8a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.427642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5ff4a-07ac-4464-974a-8d9360519d8a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.428015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5ff4a-07ac-4464-974a-8d9360519d8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.429460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5ff4a-07ac-4464-974a-8d9360519d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:10.431316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b5ff4a-07ac-4464-974a-8d9360519d8a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:10.433352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 106/215] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:10.439142 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5ff4a-07ac-4464-974a-8d9360519d8a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:10.450597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839082d-654e-46d6-879b-e5e947c5cfeb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:10.498189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616dd8b9-47d0-4423-bde8-bd9a88e3da7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:10.499152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dd8b9-47d0-4423-bde8-bd9a88e3da7a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.506886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dd8b9-47d0-4423-bde8-bd9a88e3da7a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.506886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616dd8b9-47d0-4423-bde8-bd9a88e3da7a None admin] PATH_INFO: `/v3/users/f30562f86fca41abafdc549ae74efd85` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.530114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dd8b9-47d0-4423-bde8-bd9a88e3da7a None admin] RBAC: Authorizing `identity:get_user(user_id=f30562f86fca41abafdc549ae74efd85)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:10.532052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616dd8b9-47d0-4423-bde8-bd9a88e3da7a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:10.534270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 110/216] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:10 2022] GET /identity/v3/users/f30562f86fca41abafdc549ae74efd85 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:10.536094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a6acb-06c3-4618-bbfa-1c7a8ca9177c None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.536593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a6acb-06c3-4618-bbfa-1c7a8ca9177c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.536874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a6acb-06c3-4618-bbfa-1c7a8ca9177c None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.541794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 111/217] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:10.541794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd49b2b-0774-4799-9406-b51f716af874 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.541794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd49b2b-0774-4799-9406-b51f716af874 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.541794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd49b2b-0774-4799-9406-b51f716af874 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.541794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 112/218] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:10.546273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd49b2b-0774-4799-9406-b51f716af874 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:10.565942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5ff4a-07ac-4464-974a-8d9360519d8a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:10.567490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:10.579470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:10.604316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e98d1-4acb-4f60-a67a-a6d67cf1327b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:10.605059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e98d1-4acb-4f60-a67a-a6d67cf1327b None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.605343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e98d1-4acb-4f60-a67a-a6d67cf1327b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.605698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e98d1-4acb-4f60-a67a-a6d67cf1327b None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.606377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e98d1-4acb-4f60-a67a-a6d67cf1327b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:10.607840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e98d1-4acb-4f60-a67a-a6d67cf1327b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:10.632019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 107/219] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:10 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:35:10.633279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6803e-8263-4a0a-aac9-a20c5d36d8d4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.633568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6803e-8263-4a0a-aac9-a20c5d36d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.633871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6803e-8263-4a0a-aac9-a20c5d36d8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.654920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-08a6803e-8263-4a0a-aac9-a20c5d36d8d4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:10.656161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-08a6803e-8263-4a0a-aac9-a20c5d36d8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:10.678505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd49b2b-0774-4799-9406-b51f716af874 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:10.712880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8ce89-e4ff-45f6-adae-caf0113b0cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:10.713826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ce89-e4ff-45f6-adae-caf0113b0cbb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.713982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ce89-e4ff-45f6-adae-caf0113b0cbb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.714230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8ce89-e4ff-45f6-adae-caf0113b0cbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.745727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 108/220] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:10.747605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58533f53-90dd-46f7-8fcc-b9cf96e0ad71 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:10.747906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58533f53-90dd-46f7-8fcc-b9cf96e0ad71 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:10.748175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58533f53-90dd-46f7-8fcc-b9cf96e0ad71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:10.770028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-58533f53-90dd-46f7-8fcc-b9cf96e0ad71 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:10.771621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-58533f53-90dd-46f7-8fcc-b9cf96e0ad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:10.805800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8ce89-e4ff-45f6-adae-caf0113b0cbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:10.807313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8ce89-e4ff-45f6-adae-caf0113b0cbb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:10.844268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:10.854760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 109/221] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:10.927877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 113/222] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:10.933765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58533f53-90dd-46f7-8fcc-b9cf96e0ad71 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:10.956904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:11.021977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8ce89-e4ff-45f6-adae-caf0113b0cbb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:11.065570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58533f53-90dd-46f7-8fcc-b9cf96e0ad71 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:11.115206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eede57-8fff-41de-a529-3cda6daa1ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:11.116117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eede57-8fff-41de-a529-3cda6daa1ad0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:11.116402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eede57-8fff-41de-a529-3cda6daa1ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:11.117127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eede57-8fff-41de-a529-3cda6daa1ad0 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:11.118587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eede57-8fff-41de-a529-3cda6daa1ad0 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:11.120808 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eede57-8fff-41de-a529-3cda6daa1ad0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:11.123402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 110/223] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:10 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:11.129299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eede57-8fff-41de-a529-3cda6daa1ad0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:11.174433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8ce89-e4ff-45f6-adae-caf0113b0cbb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:11.215242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:11.225412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ca7e0-9dce-4922-a7f8-8f52488681fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:11.226381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ca7e0-9dce-4922-a7f8-8f52488681fa None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:11.226805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ca7e0-9dce-4922-a7f8-8f52488681fa None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:11.227215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ca7e0-9dce-4922-a7f8-8f52488681fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:11.240095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ca7e0-9dce-4922-a7f8-8f52488681fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:11.246041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-b56ca7e0-9dce-4922-a7f8-8f52488681fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:11.249231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:11.250189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:11.251019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:11.252943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 114/224] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:11.254469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ca7e0-9dce-4922-a7f8-8f52488681fa None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:11.275347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eede57-8fff-41de-a529-3cda6daa1ad0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:11.319768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:11.320647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:11.321061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:11.321469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:11.322754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] RBAC: Adding query filter params (role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc, user.id=f30562f86fca41abafdc549ae74efd85) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:11.323165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:11.324747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:11.332101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 111/225] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/role_assignments?user.id=f30562f86fca41abafdc549ae74efd85&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:11.337878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:11.433574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ca7e0-9dce-4922-a7f8-8f52488681fa None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:11.483126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557341b0-4f7d-45e1-96ea-30c960638679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:11.483784 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557341b0-4f7d-45e1-96ea-30c960638679 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:11.484574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557341b0-4f7d-45e1-96ea-30c960638679 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:11.484849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557341b0-4f7d-45e1-96ea-30c960638679 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:11.517955 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557341b0-4f7d-45e1-96ea-30c960638679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-557341b0-4f7d-45e1-96ea-30c960638679 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:11.531167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:11.532764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:11.534233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 115/226] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:11.546505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557341b0-4f7d-45e1-96ea-30c960638679 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:11.574567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c37489-e7f3-4c49-9608-4ce9aa9ab534 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:11.622029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:11.622664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:11.622955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:11.623230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:11.623916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:11.624223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:11.625663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:11.634061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:11.649938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 112/227] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:11.658949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:11.682211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:11.737754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557341b0-4f7d-45e1-96ea-30c960638679 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:11.780043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:11.780665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:11.780943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:11.781215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:11.781926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:11.782226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:11.783632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:11.806125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 116/228] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:11.815323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:11.875304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ab0c0-1ae5-4946-afe5-548e9278ceb3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:11.933294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ba7ef-72f9-4580-84dd-2790873512f1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:11.947826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:11.954703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834b244-1db3-4132-afe6-d295fd8943f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:11.955709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834b244-1db3-4132-afe6-d295fd8943f4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:11.956167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834b244-1db3-4132-afe6-d295fd8943f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:11.956603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834b244-1db3-4132-afe6-d295fd8943f4 None admin] PATH_INFO: `/v3/users/733438bf54304476bda847bd56fd5b81` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:11.958574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834b244-1db3-4132-afe6-d295fd8943f4 None admin] RBAC: Authorizing `identity:get_user(user_id=733438bf54304476bda847bd56fd5b81)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:11.960044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834b244-1db3-4132-afe6-d295fd8943f4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:11.969882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 113/229] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/users/733438bf54304476bda847bd56fd5b81 => generated 283 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:11.975330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834b244-1db3-4132-afe6-d295fd8943f4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:11.976197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cd134-1934-4964-ba14-cabe08f82ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:11.977077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd134-1934-4964-ba14-cabe08f82ded None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:11.977953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd134-1934-4964-ba14-cabe08f82ded None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:11.978769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd134-1934-4964-ba14-cabe08f82ded None admin] PATH_INFO: `/v3/users/733438bf54304476bda847bd56fd5b81` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:11.981996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cd134-1934-4964-ba14-cabe08f82ded None admin] RBAC: Authorizing `identity:get_user(user_id=733438bf54304476bda847bd56fd5b81)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:11.983716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cd134-1934-4964-ba14-cabe08f82ded None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:11.990013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 117/230] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/users/733438bf54304476bda847bd56fd5b81 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:11.993828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cd134-1934-4964-ba14-cabe08f82ded None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:12.122602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:12.151763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834b244-1db3-4132-afe6-d295fd8943f4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:12.193225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e566d9-897c-4eac-b4bd-4b414dc8dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:12.194255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e566d9-897c-4eac-b4bd-4b414dc8dcd8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:12.194539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e566d9-897c-4eac-b4bd-4b414dc8dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:12.194815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e566d9-897c-4eac-b4bd-4b414dc8dcd8 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:12.196190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e566d9-897c-4eac-b4bd-4b414dc8dcd8 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:12.198163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e566d9-897c-4eac-b4bd-4b414dc8dcd8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:12.200342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 114/231] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:12.208253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e566d9-897c-4eac-b4bd-4b414dc8dcd8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:12.327978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e566d9-897c-4eac-b4bd-4b414dc8dcd8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:12.339949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cd134-1934-4964-ba14-cabe08f82ded None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:12.365256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abc00a-4e5b-45b3-afcc-a8d59a6311b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:12.366083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc00a-4e5b-45b3-afcc-a8d59a6311b3 None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:12.366371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc00a-4e5b-45b3-afcc-a8d59a6311b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:12.366644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc00a-4e5b-45b3-afcc-a8d59a6311b3 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/users/733438bf54304476bda847bd56fd5b81/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:12.368743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abc00a-4e5b-45b3-afcc-a8d59a6311b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9b796cfb79246dca28d73123bc5e6bc, user_id=733438bf54304476bda847bd56fd5b81, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:12.370361 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abc00a-4e5b-45b3-afcc-a8d59a6311b3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:12.394155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 115/232] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:35:12 2022] PUT /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/users/733438bf54304476bda847bd56fd5b81/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:12.416415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec058b2-44d3-4762-8226-df384931b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:12.417166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec058b2-44d3-4762-8226-df384931b805 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:12.417449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec058b2-44d3-4762-8226-df384931b805 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:12.417751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec058b2-44d3-4762-8226-df384931b805 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:12.419816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec058b2-44d3-4762-8226-df384931b805 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:12.421402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec058b2-44d3-4762-8226-df384931b805 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:12.424303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 118/233] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:11 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:12.431203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abc00a-4e5b-45b3-afcc-a8d59a6311b3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:12.450652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:12.559818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abc00a-4e5b-45b3-afcc-a8d59a6311b3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:12.600991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902eff87-db69-4187-ac00-968308d5fc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:12.601650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eff87-db69-4187-ac00-968308d5fc9a None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:12.601947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eff87-db69-4187-ac00-968308d5fc9a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:12.602269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902eff87-db69-4187-ac00-968308d5fc9a None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/users/733438bf54304476bda847bd56fd5b81/roles/90097e8ce1394daba58b200de808ad59` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:12.604747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eff87-db69-4187-ac00-968308d5fc9a None admin] RBAC: Authorizing `identity:create_grant(project_id=c9b796cfb79246dca28d73123bc5e6bc, user_id=733438bf54304476bda847bd56fd5b81, role_id=90097e8ce1394daba58b200de808ad59)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:12.606920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902eff87-db69-4187-ac00-968308d5fc9a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:12.624536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 116/234] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:35:12 2022] PUT /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/users/733438bf54304476bda847bd56fd5b81/roles/90097e8ce1394daba58b200de808ad59 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:13.599232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b896e-8f5e-42ce-95f6-d59d7f4188c0 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:13.599762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b896e-8f5e-42ce-95f6-d59d7f4188c0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:13.599762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b896e-8f5e-42ce-95f6-d59d7f4188c0 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:13.601135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 119/235] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:13.606985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46af37-5fce-45d2-b02e-5de4bf105eb3 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:13.607479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46af37-5fce-45d2-b02e-5de4bf105eb3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:13.607575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46af37-5fce-45d2-b02e-5de4bf105eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:13.634439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3f46af37-5fce-45d2-b02e-5de4bf105eb3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:13.635648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3f46af37-5fce-45d2-b02e-5de4bf105eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:13.708968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:13.715084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 117/236] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:13.912651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa17ed-8b45-44c3-b22b-fb7ab34cd722 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:13.913024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa17ed-8b45-44c3-b22b-fb7ab34cd722 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:13.913214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa17ed-8b45-44c3-b22b-fb7ab34cd722 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:13.914781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 120/237] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:13.920753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47f3a-5237-459d-8610-e41973c4b698 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:13.921039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47f3a-5237-459d-8610-e41973c4b698 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:13.921319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47f3a-5237-459d-8610-e41973c4b698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:13.944833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-23a47f3a-5237-459d-8610-e41973c4b698 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:13.946074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-23a47f3a-5237-459d-8610-e41973c4b698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:14.084521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 118/238] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:14.101106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa17ed-8b45-44c3-b22b-fb7ab34cd722 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:14.169452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:14.214312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea794b-b634-46a8-98e7-867ec18e034b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:14.214683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea794b-b634-46a8-98e7-867ec18e034b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:14.215187 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea794b-b634-46a8-98e7-867ec18e034b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:14.217120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 119/239] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:14.227516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397ba3a-9910-4277-94de-1c78e3773cf5 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:14.227809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397ba3a-9910-4277-94de-1c78e3773cf5 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:14.228073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397ba3a-9910-4277-94de-1c78e3773cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:14.264659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa17ed-8b45-44c3-b22b-fb7ab34cd722 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:14.273492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-4397ba3a-9910-4277-94de-1c78e3773cf5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:14.274737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4397ba3a-9910-4277-94de-1c78e3773cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:14.330262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1e9f-0506-49dc-9bf0-95b965a4cd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:14.330908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1e9f-0506-49dc-9bf0-95b965a4cd1d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:14.331184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1e9f-0506-49dc-9bf0-95b965a4cd1d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:14.331449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1e9f-0506-49dc-9bf0-95b965a4cd1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:14.350368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce1e9f-0506-49dc-9bf0-95b965a4cd1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-d0ce1e9f-0506-49dc-9bf0-95b965a4cd1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:14.355805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:14.356798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:14.357620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 121/240] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:14.366937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1e9f-0506-49dc-9bf0-95b965a4cd1d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:14.417965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 120/241] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:14.454504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397ba3a-9910-4277-94de-1c78e3773cf5 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:14.529357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:14.602065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1e9f-0506-49dc-9bf0-95b965a4cd1d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:14.607623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397ba3a-9910-4277-94de-1c78e3773cf5 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:14.619554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:14.654439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:14.655181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:14.655459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:14.655727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:14.656412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:14.656712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:14.658007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758f37b-f6d8-4abf-aa63-3ec41826e849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:14.658231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:14.659067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758f37b-f6d8-4abf-aa63-3ec41826e849 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:14.659561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758f37b-f6d8-4abf-aa63-3ec41826e849 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:14.660032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758f37b-f6d8-4abf-aa63-3ec41826e849 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:14.661368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758f37b-f6d8-4abf-aa63-3ec41826e849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:14.663072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758f37b-f6d8-4abf-aa63-3ec41826e849 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:14.665183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 121/242] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:14.675926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758f37b-f6d8-4abf-aa63-3ec41826e849 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:14.677964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 122/243] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:14.689554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:14.807303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f3506-9987-4fed-b116-00337b89a526 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:14.820017 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758f37b-f6d8-4abf-aa63-3ec41826e849 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:14.848358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:14.857776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fe21c-3592-49d1-a041-1f6d17c53bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:14.858858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fe21c-3592-49d1-a041-1f6d17c53bb8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:14.859341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fe21c-3592-49d1-a041-1f6d17c53bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:14.859825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fe21c-3592-49d1-a041-1f6d17c53bb8 None admin] PATH_INFO: `/v3/users/1da32f4a120b485e98dde2636d778741` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:14.863284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fb2c2-53b4-4d9e-897b-51e1c404f246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:14.864098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fb2c2-53b4-4d9e-897b-51e1c404f246 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:14.864523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fb2c2-53b4-4d9e-897b-51e1c404f246 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:14.864965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7fb2c2-53b4-4d9e-897b-51e1c404f246 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:14.865805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fb2c2-53b4-4d9e-897b-51e1c404f246 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:14.867438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7fb2c2-53b4-4d9e-897b-51e1c404f246 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:14.879559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fe21c-3592-49d1-a041-1f6d17c53bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=1da32f4a120b485e98dde2636d778741)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:14.881274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fe21c-3592-49d1-a041-1f6d17c53bb8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:14.883736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 123/244] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:14 2022] GET /identity/v3/users/1da32f4a120b485e98dde2636d778741 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:14.884671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fe21c-3592-49d1-a041-1f6d17c53bb8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:14.893676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 122/245] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:14 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:14.895009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fb2c2-53b4-4d9e-897b-51e1c404f246 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:14.926065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:15.016347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7fb2c2-53b4-4d9e-897b-51e1c404f246 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:15.055500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fe21c-3592-49d1-a041-1f6d17c53bb8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:15.060305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b846ec-e30f-448b-b9c7-0093656ee571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:15.061038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b846ec-e30f-448b-b9c7-0093656ee571 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.061423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b846ec-e30f-448b-b9c7-0093656ee571 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.061809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b846ec-e30f-448b-b9c7-0093656ee571 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.129765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:15.141841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e323-b5b5-4d70-9215-61783e144945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:15.142950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e323-b5b5-4d70-9215-61783e144945 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.143428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e323-b5b5-4d70-9215-61783e144945 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.144046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454e323-b5b5-4d70-9215-61783e144945 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.151987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b846ec-e30f-448b-b9c7-0093656ee571 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:15.154102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b846ec-e30f-448b-b9c7-0093656ee571 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:15.159769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454e323-b5b5-4d70-9215-61783e144945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-8454e323-b5b5-4d70-9215-61783e144945 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:15.165040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:15.166013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:15.166814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:15.169757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 124/246] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:15.184904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e323-b5b5-4d70-9215-61783e144945 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:15.318157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:15.343168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 123/247] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:15.343980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454e323-b5b5-4d70-9215-61783e144945 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:15.349400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b846ec-e30f-448b-b9c7-0093656ee571 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:15.406561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239e951-a8b4-4660-a0a4-a20be1c5976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:15.407541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e951-a8b4-4660-a0a4-a20be1c5976c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.407816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e951-a8b4-4660-a0a4-a20be1c5976c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.408081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e951-a8b4-4660-a0a4-a20be1c5976c None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.408763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e951-a8b4-4660-a0a4-a20be1c5976c None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:15.409139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e951-a8b4-4660-a0a4-a20be1c5976c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:15.411691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e951-a8b4-4660-a0a4-a20be1c5976c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:15.424272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 125/248] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:15.425723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948d28-dda0-406d-8797-044a52e33b2a None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.426009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948d28-dda0-406d-8797-044a52e33b2a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.426277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948d28-dda0-406d-8797-044a52e33b2a None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.431969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 126/249] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:15.434685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dce7c3-1538-4dcb-ba63-d895210055dc None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.437036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dce7c3-1538-4dcb-ba63-d895210055dc None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.437373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dce7c3-1538-4dcb-ba63-d895210055dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.462996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-00dce7c3-1538-4dcb-ba63-d895210055dc None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:15.464536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-00dce7c3-1538-4dcb-ba63-d895210055dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:15.500443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:15.539794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b846ec-e30f-448b-b9c7-0093656ee571 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:15.564668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 127/250] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:15.566250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dce7c3-1538-4dcb-ba63-d895210055dc None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:15.588755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30644f09-73ce-4622-8254-f48fc15d0934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:15.589953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644f09-73ce-4622-8254-f48fc15d0934 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.590472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644f09-73ce-4622-8254-f48fc15d0934 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.590902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30644f09-73ce-4622-8254-f48fc15d0934 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.593374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30644f09-73ce-4622-8254-f48fc15d0934 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:15.598406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30644f09-73ce-4622-8254-f48fc15d0934 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:15.601318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 124/251] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:15 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:15.603527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9767-8332-48aa-9a90-7964345cb653 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.604023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9767-8332-48aa-9a90-7964345cb653 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.604515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9767-8332-48aa-9a90-7964345cb653 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.607849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 125/252] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:15.609893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9767-8332-48aa-9a90-7964345cb653 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:15.726770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:15.767245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dce7c3-1538-4dcb-ba63-d895210055dc None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:15.801103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9767-8332-48aa-9a90-7964345cb653 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:15.847512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8183f-8f5c-477c-8a75-dbb261471eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:15.848040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8183f-8f5c-477c-8a75-dbb261471eed None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.848650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8183f-8f5c-477c-8a75-dbb261471eed None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.848650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8183f-8f5c-477c-8a75-dbb261471eed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.849304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8183f-8f5c-477c-8a75-dbb261471eed None admin] RBAC: Adding query filter params (role.id=935445ed062541ceb33d5021be2b092a, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc, user.id=1da32f4a120b485e98dde2636d778741) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:15.849899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de6fc1-f1d1-4aac-a910-c2bcaaae0fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:15.850018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8183f-8f5c-477c-8a75-dbb261471eed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:15.851612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de6fc1-f1d1-4aac-a910-c2bcaaae0fca None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.851612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de6fc1-f1d1-4aac-a910-c2bcaaae0fca None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.851612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de6fc1-f1d1-4aac-a910-c2bcaaae0fca None admin] PATH_INFO: `/v3/users/f30562f86fca41abafdc549ae74efd85` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.853751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8183f-8f5c-477c-8a75-dbb261471eed None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:15.855201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de6fc1-f1d1-4aac-a910-c2bcaaae0fca None admin] RBAC: Authorizing `identity:get_user(user_id=f30562f86fca41abafdc549ae74efd85)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:15.855837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de6fc1-f1d1-4aac-a910-c2bcaaae0fca None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:15.861860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 126/253] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:15 2022] GET /identity/v3/role_assignments?user.id=1da32f4a120b485e98dde2636d778741&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=935445ed062541ceb33d5021be2b092a => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:15.862249 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 128/254] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:15 2022] GET /identity/v3/users/f30562f86fca41abafdc549ae74efd85 => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:15.864481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0748b-5532-45eb-a767-2097eead1666 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:15.864832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0748b-5532-45eb-a767-2097eead1666 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:15.865116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0748b-5532-45eb-a767-2097eead1666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:15.868828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de6fc1-f1d1-4aac-a910-c2bcaaae0fca None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:15.930881 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-17d0748b-5532-45eb-a767-2097eead1666 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:15.932254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-17d0748b-5532-45eb-a767-2097eead1666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:15.962561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:16.164928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 127/255] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:16.165962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0748b-5532-45eb-a767-2097eead1666 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:16.181461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de6fc1-f1d1-4aac-a910-c2bcaaae0fca None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:16.273115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938baa8-a873-482f-87df-b825d5a0be30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:16.273871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938baa8-a873-482f-87df-b825d5a0be30 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:16.274157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938baa8-a873-482f-87df-b825d5a0be30 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:16.274437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938baa8-a873-482f-87df-b825d5a0be30 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:16.276026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938baa8-a873-482f-87df-b825d5a0be30 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:16.279877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938baa8-a873-482f-87df-b825d5a0be30 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:16.283596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 129/256] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:15 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:16.290923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938baa8-a873-482f-87df-b825d5a0be30 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:16.352979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:16.497984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0748b-5532-45eb-a767-2097eead1666 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:16.578538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa464434-246e-4aa2-9543-c72ec6730e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:16.581194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464434-246e-4aa2-9543-c72ec6730e5a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:16.583052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464434-246e-4aa2-9543-c72ec6730e5a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:16.583958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464434-246e-4aa2-9543-c72ec6730e5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:16.614149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa464434-246e-4aa2-9543-c72ec6730e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-aa464434-246e-4aa2-9543-c72ec6730e5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:16.622313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:16.623873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:16.625179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:16.629738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 128/257] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 464 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:16.642988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa464434-246e-4aa2-9543-c72ec6730e5a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:16.648762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938baa8-a873-482f-87df-b825d5a0be30 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:16.657821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:16.773226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b0f8-1aea-4be7-b09c-043e194bc94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:16.774516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b0f8-1aea-4be7-b09c-043e194bc94f None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:16.775784 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b0f8-1aea-4be7-b09c-043e194bc94f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:16.776227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374b0f8-1aea-4be7-b09c-043e194bc94f None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/users/f30562f86fca41abafdc549ae74efd85/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:16.779189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374b0f8-1aea-4be7-b09c-043e194bc94f None admin] RBAC: Authorizing `identity:create_grant(project_id=c9b796cfb79246dca28d73123bc5e6bc, user_id=f30562f86fca41abafdc549ae74efd85, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:16.788127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374b0f8-1aea-4be7-b09c-043e194bc94f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:16.814250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 130/258] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:35:16 2022] PUT /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/users/f30562f86fca41abafdc549ae74efd85/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:16.815957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b0f8-1aea-4be7-b09c-043e194bc94f None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:16.912779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa464434-246e-4aa2-9543-c72ec6730e5a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:16.978587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711269f-2a3e-43f7-9ca8-4936b707e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:16.979734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711269f-2a3e-43f7-9ca8-4936b707e981 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:16.980165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711269f-2a3e-43f7-9ca8-4936b707e981 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:16.980611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711269f-2a3e-43f7-9ca8-4936b707e981 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:16.994940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711269f-2a3e-43f7-9ca8-4936b707e981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-2711269f-2a3e-43f7-9ca8-4936b707e981 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:17.000678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:17.001682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:17.002544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:17.005132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 129/259] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:17.023428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711269f-2a3e-43f7-9ca8-4936b707e981 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:17.050128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:17.104068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374b0f8-1aea-4be7-b09c-043e194bc94f None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:17.120528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:17.180176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711269f-2a3e-43f7-9ca8-4936b707e981 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:17.210454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:17.211131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:17.211413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:17.211689 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:17.212388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:17.212695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:17.214122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:17.231540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:17.232231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 131/260] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:17.232782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:17.233187 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:17.233711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:17.234518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:17.234926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:17.236431 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:17.239105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:17.251926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 130/261] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:17.253726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:17.364592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4943b-59ef-4baf-812a-da7907e5ebf2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:17.393564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:17.404047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c47618-7326-499e-9a2e-d7fb52a66b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:17.405582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c47618-7326-499e-9a2e-d7fb52a66b93 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:17.406193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c47618-7326-499e-9a2e-d7fb52a66b93 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:17.406666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c47618-7326-499e-9a2e-d7fb52a66b93 None admin] PATH_INFO: `/v3/users/733438bf54304476bda847bd56fd5b81` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:17.408093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c47618-7326-499e-9a2e-d7fb52a66b93 None admin] RBAC: Authorizing `identity:get_user(user_id=733438bf54304476bda847bd56fd5b81)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:17.409811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c47618-7326-499e-9a2e-d7fb52a66b93 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:17.416391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 131/262] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:17 2022] GET /identity/v3/users/733438bf54304476bda847bd56fd5b81 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:17.417979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c47618-7326-499e-9a2e-d7fb52a66b93 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:17.546966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c47618-7326-499e-9a2e-d7fb52a66b93 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:17.567644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81cb70-73c8-4b34-a340-bf7d5772ca2a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:17.602379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fc463-d079-4ea5-a824-7bd829578653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:17.603600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fc463-d079-4ea5-a824-7bd829578653 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:17.604133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fc463-d079-4ea5-a824-7bd829578653 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:17.604833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fc463-d079-4ea5-a824-7bd829578653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:17.652214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e951603-154e-4f54-917c-88c96c3e879d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:17.652829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951603-154e-4f54-917c-88c96c3e879d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:17.653109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951603-154e-4f54-917c-88c96c3e879d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:17.653382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951603-154e-4f54-917c-88c96c3e879d None admin] PATH_INFO: `/v3/users/733438bf54304476bda847bd56fd5b81` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:17.658415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e951603-154e-4f54-917c-88c96c3e879d None admin] RBAC: Authorizing `identity:get_user(user_id=733438bf54304476bda847bd56fd5b81)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:17.659914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e951603-154e-4f54-917c-88c96c3e879d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:17.662026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 132/263] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:17 2022] GET /identity/v3/users/733438bf54304476bda847bd56fd5b81 => generated 283 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:17.667473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e951603-154e-4f54-917c-88c96c3e879d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:17.691862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:17.710013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:17.714218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fc463-d079-4ea5-a824-7bd829578653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:17.716186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fc463-d079-4ea5-a824-7bd829578653 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:17.952440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e951603-154e-4f54-917c-88c96c3e879d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:17.981948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 132/264] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:17.984970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fc463-d079-4ea5-a824-7bd829578653 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:18.038404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bece54-3617-45a7-a94e-8d06cef23835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:18.039046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bece54-3617-45a7-a94e-8d06cef23835 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:18.039334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bece54-3617-45a7-a94e-8d06cef23835 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:18.039608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bece54-3617-45a7-a94e-8d06cef23835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:18.133034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fc463-d079-4ea5-a824-7bd829578653 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:18.138154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:18.176861 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bece54-3617-45a7-a94e-8d06cef23835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:18.178434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bece54-3617-45a7-a94e-8d06cef23835 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:18.194462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01271b93-77ad-42e9-9300-17388f073234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:18.195497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01271b93-77ad-42e9-9300-17388f073234 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:18.195973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:18.196938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01271b93-77ad-42e9-9300-17388f073234 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:18.197688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01271b93-77ad-42e9-9300-17388f073234 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:18.199151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01271b93-77ad-42e9-9300-17388f073234 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:18.201313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01271b93-77ad-42e9-9300-17388f073234 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:18.210291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 133/265] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:17 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:18.217045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01271b93-77ad-42e9-9300-17388f073234 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:18.397416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01271b93-77ad-42e9-9300-17388f073234 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:18.416677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 133/266] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:35:18.417806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8cc3c-4ba1-47b9-969d-2753967d02b3 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:18.418077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8cc3c-4ba1-47b9-969d-2753967d02b3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:18.419077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8cc3c-4ba1-47b9-969d-2753967d02b3 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:18.420387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 134/267] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:18.430675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8cc3c-4ba1-47b9-969d-2753967d02b3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:18.452296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec3ad4-59bf-4092-aa61-7ff55865b9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:18.453167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec3ad4-59bf-4092-aa61-7ff55865b9b4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:18.453671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec3ad4-59bf-4092-aa61-7ff55865b9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:18.454090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec3ad4-59bf-4092-aa61-7ff55865b9b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:18.455048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec3ad4-59bf-4092-aa61-7ff55865b9b4 None admin] RBAC: Adding query filter params (role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc, user.id=733438bf54304476bda847bd56fd5b81) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:18.455504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec3ad4-59bf-4092-aa61-7ff55865b9b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:18.457127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec3ad4-59bf-4092-aa61-7ff55865b9b4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:18.466408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 134/268] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:18 2022] GET /identity/v3/role_assignments?user.id=733438bf54304476bda847bd56fd5b81&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=90097e8ce1394daba58b200de808ad59 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:18.467990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93878799-6c3d-4cc0-9c30-f160d401d69b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:18.468436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93878799-6c3d-4cc0-9c30-f160d401d69b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:18.468857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93878799-6c3d-4cc0-9c30-f160d401d69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:18.497638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-93878799-6c3d-4cc0-9c30-f160d401d69b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:18.499303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-93878799-6c3d-4cc0-9c30-f160d401d69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:18.539709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:18.633284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 135/269] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:18.690224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113be48-72e8-4b43-80fc-588315379cc8 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:18.690799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113be48-72e8-4b43-80fc-588315379cc8 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:18.691289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113be48-72e8-4b43-80fc-588315379cc8 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:18.694369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 136/270] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:18.699562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed747683-6848-4211-92b7-7bd48a2becfd None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:18.700182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed747683-6848-4211-92b7-7bd48a2becfd None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:18.700654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed747683-6848-4211-92b7-7bd48a2becfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:18.721216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ed747683-6848-4211-92b7-7bd48a2becfd None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:18.723502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ed747683-6848-4211-92b7-7bd48a2becfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:18.735089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8cc3c-4ba1-47b9-969d-2753967d02b3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:18.749674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:18.813526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 137/271] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:18.831373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0407f-f36b-46a3-9dad-2a79c1768d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:18.832028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0407f-f36b-46a3-9dad-2a79c1768d03 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:18.832304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0407f-f36b-46a3-9dad-2a79c1768d03 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:18.832574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0407f-f36b-46a3-9dad-2a79c1768d03 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:18.838415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0407f-f36b-46a3-9dad-2a79c1768d03 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:18.839898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0407f-f36b-46a3-9dad-2a79c1768d03 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:18.842191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 135/272] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:18 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:18.852342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed747683-6848-4211-92b7-7bd48a2becfd None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:18.977228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:18.997045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed747683-6848-4211-92b7-7bd48a2becfd None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:19.027077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0407f-f36b-46a3-9dad-2a79c1768d03 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:19.074340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:19.075026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:19.075306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:19.075588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:19.076339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc, user.id=733438bf54304476bda847bd56fd5b81) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:19.076648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:19.078201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:19.087353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 138/273] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:18 2022] GET /identity/v3/role_assignments?user.id=733438bf54304476bda847bd56fd5b81&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=b698800ceec3460a8519ea627650da0f => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:19.143077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0407f-f36b-46a3-9dad-2a79c1768d03 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:19.183052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe12ccd-94d6-4149-975c-5e75ce9d5097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:19.183704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12ccd-94d6-4149-975c-5e75ce9d5097 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:19.184088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12ccd-94d6-4149-975c-5e75ce9d5097 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:19.184364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12ccd-94d6-4149-975c-5e75ce9d5097 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:19.197218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:19.197410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe12ccd-94d6-4149-975c-5e75ce9d5097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-bbe12ccd-94d6-4149-975c-5e75ce9d5097 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:19.203063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:19.204049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:19.204850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:19.209439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 136/274] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:19.210460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe12ccd-94d6-4149-975c-5e75ce9d5097 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:19.255659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:19.355239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc730b2-e4fc-46a6-bec6-4fdff02a53b8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:19.398136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe12ccd-94d6-4149-975c-5e75ce9d5097 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:19.416287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5ff4d-70b9-4f21-9ddd-9e76d8d8e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:19.416947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5ff4d-70b9-4f21-9ddd-9e76d8d8e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:19.417325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5ff4d-70b9-4f21-9ddd-9e76d8d8e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:19.417627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5ff4d-70b9-4f21-9ddd-9e76d8d8e3a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:19.420354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5ff4d-70b9-4f21-9ddd-9e76d8d8e3a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:19.421434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f5ff4d-70b9-4f21-9ddd-9e76d8d8e3a9 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:19.423429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 139/275] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:19.424525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3140efe-f28c-4349-b225-5f1eaa8d6a81 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:19.424812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3140efe-f28c-4349-b225-5f1eaa8d6a81 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:19.425087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3140efe-f28c-4349-b225-5f1eaa8d6a81 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:19.427752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 140/276] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:19.428854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3140efe-f28c-4349-b225-5f1eaa8d6a81 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:19.441456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279de5b5-9337-47a0-ad89-ed7e2acf94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:19.442455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279de5b5-9337-47a0-ad89-ed7e2acf94de None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:19.442902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279de5b5-9337-47a0-ad89-ed7e2acf94de None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:19.443336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279de5b5-9337-47a0-ad89-ed7e2acf94de None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:19.444462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279de5b5-9337-47a0-ad89-ed7e2acf94de None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:19.444990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279de5b5-9337-47a0-ad89-ed7e2acf94de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:19.447627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279de5b5-9337-47a0-ad89-ed7e2acf94de None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:19.458653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:19.463031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 137/277] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:19.465775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf0d0e-25d3-4bb7-9190-80fde624f1d6 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:19.466246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf0d0e-25d3-4bb7-9190-80fde624f1d6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:19.466692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf0d0e-25d3-4bb7-9190-80fde624f1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:19.494004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-7ebf0d0e-25d3-4bb7-9190-80fde624f1d6 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:19.495478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7ebf0d0e-25d3-4bb7-9190-80fde624f1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:19.572697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3140efe-f28c-4349-b225-5f1eaa8d6a81 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:19.592394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 138/278] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:19.593166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf0d0e-25d3-4bb7-9190-80fde624f1d6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:19.623028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33510286-6162-44ef-a54f-a633d103c99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:19.623874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33510286-6162-44ef-a54f-a633d103c99f None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:19.624162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33510286-6162-44ef-a54f-a633d103c99f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:19.624437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33510286-6162-44ef-a54f-a633d103c99f None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:19.625151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33510286-6162-44ef-a54f-a633d103c99f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:19.627601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33510286-6162-44ef-a54f-a633d103c99f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:19.636327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:19.669642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 141/279] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:19 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:19.748062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf0d0e-25d3-4bb7-9190-80fde624f1d6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:19.796822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82458f49-15d6-465a-9514-fc0974d4e56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:19.797490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82458f49-15d6-465a-9514-fc0974d4e56b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:19.797938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82458f49-15d6-465a-9514-fc0974d4e56b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:19.798208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82458f49-15d6-465a-9514-fc0974d4e56b None admin] PATH_INFO: `/v3/users/b3fbbdf3d25544738102b678430f0c10` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:19.815885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82458f49-15d6-465a-9514-fc0974d4e56b None admin] RBAC: Authorizing `identity:get_user(user_id=b3fbbdf3d25544738102b678430f0c10)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:19.817374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82458f49-15d6-465a-9514-fc0974d4e56b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:19.819221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 139/280] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:19 2022] GET /identity/v3/users/b3fbbdf3d25544738102b678430f0c10 => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:19.826369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33510286-6162-44ef-a54f-a633d103c99f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:19.894865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82458f49-15d6-465a-9514-fc0974d4e56b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:19.936090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:19.975105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33510286-6162-44ef-a54f-a633d103c99f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:20.002229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:20.029447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82458f49-15d6-465a-9514-fc0974d4e56b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:20.031428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c4b13-2c31-4ba2-83e9-3033d1b5a00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:20.032729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4b13-2c31-4ba2-83e9-3033d1b5a00d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.032729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4b13-2c31-4ba2-83e9-3033d1b5a00d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.032729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c4b13-2c31-4ba2-83e9-3033d1b5a00d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.034193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c4b13-2c31-4ba2-83e9-3033d1b5a00d None admin] RBAC: Adding query filter params (role.id=afc24e28d6c34708ab897fc4b7029bb2, user.id=b3fbbdf3d25544738102b678430f0c10, scope.system=all) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:20.034193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c4b13-2c31-4ba2-83e9-3033d1b5a00d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:20.035466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c4b13-2c31-4ba2-83e9-3033d1b5a00d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:20.043446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 142/281] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Sep 13 19:35:19 2022] GET /identity/v3/role_assignments?user.id=b3fbbdf3d25544738102b678430f0c10&scope.system=all&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:20.045886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b7f57-0be3-41d7-a7a3-12b84cd54729 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.046298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b7f57-0be3-41d7-a7a3-12b84cd54729 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.046583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b7f57-0be3-41d7-a7a3-12b84cd54729 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.048268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 143/282] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:20.054262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faf18e-3d98-40dd-a699-a21b6c8ba89d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.054594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faf18e-3d98-40dd-a699-a21b6c8ba89d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.054815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9faf18e-3d98-40dd-a699-a21b6c8ba89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.076611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b9faf18e-3d98-40dd-a699-a21b6c8ba89d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:20.077837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b9faf18e-3d98-40dd-a699-a21b6c8ba89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:20.082188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b6cf6-59c9-411e-99be-542faf31911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:20.083166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b6cf6-59c9-411e-99be-542faf31911a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.083442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b6cf6-59c9-411e-99be-542faf31911a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.083714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b6cf6-59c9-411e-99be-542faf31911a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.100019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b6cf6-59c9-411e-99be-542faf31911a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-024b6cf6-59c9-411e-99be-542faf31911a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:20.105895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:20.106852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:20.107683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:20.108951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 140/283] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:20.114602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b6cf6-59c9-411e-99be-542faf31911a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:20.187272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 144/284] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:20.265440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b6cf6-59c9-411e-99be-542faf31911a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:20.304637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:20.305555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.306051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.306473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.307304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:20.307756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:20.309860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:20.319801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 141/285] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:20.326830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9faf18e-3d98-40dd-a699-a21b6c8ba89d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:20.431470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:20.471300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9faf18e-3d98-40dd-a699-a21b6c8ba89d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:20.490576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:20.508768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:20.509112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334a71f-424c-4dff-913f-a1bf3ee52635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:20.509783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334a71f-424c-4dff-913f-a1bf3ee52635 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.510069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334a71f-424c-4dff-913f-a1bf3ee52635 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.510340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334a71f-424c-4dff-913f-a1bf3ee52635 None admin] PATH_INFO: `/v3/users/1da32f4a120b485e98dde2636d778741` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.511667 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334a71f-424c-4dff-913f-a1bf3ee52635 None admin] RBAC: Authorizing `identity:get_user(user_id=1da32f4a120b485e98dde2636d778741)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:20.513367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334a71f-424c-4dff-913f-a1bf3ee52635 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:20.515272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 145/286] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:20 2022] GET /identity/v3/users/1da32f4a120b485e98dde2636d778741 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:20.520488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334a71f-424c-4dff-913f-a1bf3ee52635 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:20.648203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cabe3-6731-4d49-8ac3-075d1fbe4d5a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:20.691562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334a71f-424c-4dff-913f-a1bf3ee52635 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:20.730463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd0527-c918-4991-8278-a3a7c3e9ffac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:20.731177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd0527-c918-4991-8278-a3a7c3e9ffac None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.731459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd0527-c918-4991-8278-a3a7c3e9ffac None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.731562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63899b08-90dc-4915-9ddd-b5b556f20dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:20.731760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd0527-c918-4991-8278-a3a7c3e9ffac None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.732199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899b08-90dc-4915-9ddd-b5b556f20dcd None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.732474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899b08-90dc-4915-9ddd-b5b556f20dcd None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.732742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63899b08-90dc-4915-9ddd-b5b556f20dcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.733047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd0527-c918-4991-8278-a3a7c3e9ffac None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:20.734534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd0527-c918-4991-8278-a3a7c3e9ffac None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:20.736332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 146/287] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:20 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:20.742337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd0527-c918-4991-8278-a3a7c3e9ffac None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:20.746218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:20.766301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63899b08-90dc-4915-9ddd-b5b556f20dcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-63899b08-90dc-4915-9ddd-b5b556f20dcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:20.771851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:20.772797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:20.774082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 142/288] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:20.780986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63899b08-90dc-4915-9ddd-b5b556f20dcd None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:20.860608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd0527-c918-4991-8278-a3a7c3e9ffac None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:20.902492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dc00d-a11f-4ff4-a166-cba4051f9d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:20.903309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc00d-a11f-4ff4-a166-cba4051f9d9f None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:20.903700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc00d-a11f-4ff4-a166-cba4051f9d9f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:20.903999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29dc00d-a11f-4ff4-a166-cba4051f9d9f None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/users/1da32f4a120b485e98dde2636d778741/roles/935445ed062541ceb33d5021be2b092a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:20.912312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dc00d-a11f-4ff4-a166-cba4051f9d9f None admin] RBAC: Authorizing `identity:create_grant(project_id=c9b796cfb79246dca28d73123bc5e6bc, user_id=1da32f4a120b485e98dde2636d778741, role_id=935445ed062541ceb33d5021be2b092a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:20.913900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29dc00d-a11f-4ff4-a166-cba4051f9d9f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:20.939575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 147/289] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:35:20 2022] PUT /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/users/1da32f4a120b485e98dde2636d778741/roles/935445ed062541ceb33d5021be2b092a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:20.962757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:20.993398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63899b08-90dc-4915-9ddd-b5b556f20dcd None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:21.041458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:21.042208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:21.042441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:21.042657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:21.043505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:21.043811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:21.045349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:21.055112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 143/290] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:21.063064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dc00d-a11f-4ff4-a166-cba4051f9d9f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:21.167549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:21.223885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29dc00d-a11f-4ff4-a166-cba4051f9d9f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:21.274949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4896a2-ef50-4da2-ad15-2bd5e97c2351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:21.275845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4896a2-ef50-4da2-ad15-2bd5e97c2351 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:21.276180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4896a2-ef50-4da2-ad15-2bd5e97c2351 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:21.276570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4896a2-ef50-4da2-ad15-2bd5e97c2351 None admin] PATH_INFO: `/v3/users/f30562f86fca41abafdc549ae74efd85` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:21.278268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4896a2-ef50-4da2-ad15-2bd5e97c2351 None admin] RBAC: Authorizing `identity:get_user(user_id=f30562f86fca41abafdc549ae74efd85)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:21.280310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4896a2-ef50-4da2-ad15-2bd5e97c2351 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:21.283126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 148/291] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:21 2022] GET /identity/v3/users/f30562f86fca41abafdc549ae74efd85 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:21.291796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:21.419848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b92aa-c9c7-4661-957f-4bae9f69908e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:21.456236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:21.464733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf81a4-3677-40bf-a184-61f30c8486de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:21.465625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf81a4-3677-40bf-a184-61f30c8486de None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:21.470022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf81a4-3677-40bf-a184-61f30c8486de None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:21.470315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf81a4-3677-40bf-a184-61f30c8486de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:21.571782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf81a4-3677-40bf-a184-61f30c8486de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:21.573517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf81a4-3677-40bf-a184-61f30c8486de None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:21.680593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 144/292] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:21.687107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4896a2-ef50-4da2-ad15-2bd5e97c2351 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:21.828400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4896a2-ef50-4da2-ad15-2bd5e97c2351 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:21.884432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:21.892090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002b633-71c3-4265-bde6-babe626cfe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:21.892745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002b633-71c3-4265-bde6-babe626cfe64 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:21.893031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002b633-71c3-4265-bde6-babe626cfe64 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:21.893307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002b633-71c3-4265-bde6-babe626cfe64 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:21.894510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002b633-71c3-4265-bde6-babe626cfe64 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:21.896107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002b633-71c3-4265-bde6-babe626cfe64 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:21.897816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 149/293] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:21 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:21.904577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf81a4-3677-40bf-a184-61f30c8486de None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:21.949253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:22.058581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf81a4-3677-40bf-a184-61f30c8486de None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:22.113883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24747-c8db-4618-8cbe-f097122b6434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:22.114873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24747-c8db-4618-8cbe-f097122b6434 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:22.115346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24747-c8db-4618-8cbe-f097122b6434 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:22.115827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24747-c8db-4618-8cbe-f097122b6434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:22.116770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a24747-c8db-4618-8cbe-f097122b6434 None admin] RBAC: Adding query filter params (role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc, user.id=f30562f86fca41abafdc549ae74efd85) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:22.117291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a24747-c8db-4618-8cbe-f097122b6434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:22.119112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a24747-c8db-4618-8cbe-f097122b6434 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:22.130839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 145/294] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:21 2022] GET /identity/v3/role_assignments?user.id=f30562f86fca41abafdc549ae74efd85&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:22.603918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62957e99-310f-4491-88d5-54575bb25b54 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:22.603918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62957e99-310f-4491-88d5-54575bb25b54 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:22.603918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62957e99-310f-4491-88d5-54575bb25b54 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:22.604704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 150/295] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:22.609459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0ec5b-b80c-4586-bbb0-7d24a0928b62 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:22.610123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0ec5b-b80c-4586-bbb0-7d24a0928b62 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:22.610620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0ec5b-b80c-4586-bbb0-7d24a0928b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:22.636036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-1fa0ec5b-b80c-4586-bbb0-7d24a0928b62 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:22.637299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1fa0ec5b-b80c-4586-bbb0-7d24a0928b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:22.731746 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 146/296] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:22.872555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10722eb7-1f1e-4c35-8d45-4883d0828206 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:22.873370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10722eb7-1f1e-4c35-8d45-4883d0828206 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:22.873996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10722eb7-1f1e-4c35-8d45-4883d0828206 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:22.876211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 151/297] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:22.882386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14590fb0-a408-45c1-af73-7db924792a13 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:22.882979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14590fb0-a408-45c1-af73-7db924792a13 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:22.883475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14590fb0-a408-45c1-af73-7db924792a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:22.904083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-14590fb0-a408-45c1-af73-7db924792a13 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:22.906371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-14590fb0-a408-45c1-af73-7db924792a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:22.929848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:22.989781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 147/298] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:23.069686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10722eb7-1f1e-4c35-8d45-4883d0828206 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:23.172742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76154998-b867-4c5f-99dd-e9dd60f16a49 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.173752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76154998-b867-4c5f-99dd-e9dd60f16a49 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.174232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76154998-b867-4c5f-99dd-e9dd60f16a49 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.176088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 148/299] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:23.181576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9acc-cd9f-4a9d-91b8-ac15801e700c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.182171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9acc-cd9f-4a9d-91b8-ac15801e700c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.182667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d9acc-cd9f-4a9d-91b8-ac15801e700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.193499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10722eb7-1f1e-4c35-8d45-4883d0828206 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:23.202976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-954d9acc-cd9f-4a9d-91b8-ac15801e700c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:23.204626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-954d9acc-cd9f-4a9d-91b8-ac15801e700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:23.237515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df1639-1ceb-4e48-ae7a-08ab72020870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:23.238596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df1639-1ceb-4e48-ae7a-08ab72020870 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.239080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df1639-1ceb-4e48-ae7a-08ab72020870 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.239568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df1639-1ceb-4e48-ae7a-08ab72020870 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.257145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df1639-1ceb-4e48-ae7a-08ab72020870 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-61df1639-1ceb-4e48-ae7a-08ab72020870 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:23.263245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:23.264979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:23.266127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:23.268873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 152/300] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:23.271977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df1639-1ceb-4e48-ae7a-08ab72020870 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:23.305786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 149/301] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:23.307121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d9acc-cd9f-4a9d-91b8-ac15801e700c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:23.349973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:23.395257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df1639-1ceb-4e48-ae7a-08ab72020870 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:23.429148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:23.436418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cf0ed-caf4-4266-9841-638bf5ca707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:23.437359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cf0ed-caf4-4266-9841-638bf5ca707b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.437925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cf0ed-caf4-4266-9841-638bf5ca707b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.438517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cf0ed-caf4-4266-9841-638bf5ca707b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.439843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cf0ed-caf4-4266-9841-638bf5ca707b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:23.441583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cf0ed-caf4-4266-9841-638bf5ca707b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:23.448661 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 153/302] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:23.450401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cf0ed-caf4-4266-9841-638bf5ca707b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:23.451529 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d9acc-cd9f-4a9d-91b8-ac15801e700c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:23.495411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:23.496781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.496987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.497268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.497980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:23.498289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:23.499998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:23.512864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 150/303] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:23.518966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:23.578831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cf0ed-caf4-4266-9841-638bf5ca707b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:23.607646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:23.617046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc517e3-cb99-49b8-8b5b-ce7dc57bde8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:23.618072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc517e3-cb99-49b8-8b5b-ce7dc57bde8e None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.618503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc517e3-cb99-49b8-8b5b-ce7dc57bde8e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.619081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc517e3-cb99-49b8-8b5b-ce7dc57bde8e None admin] PATH_INFO: `/v3/groups` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.619736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc517e3-cb99-49b8-8b5b-ce7dc57bde8e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:23.621295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc517e3-cb99-49b8-8b5b-ce7dc57bde8e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:23.644474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 154/304] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Sep 13 19:35:23 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:35:23.644474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc517e3-cb99-49b8-8b5b-ce7dc57bde8e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:23.652911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e318e9-6208-4e6e-ba8f-b62d978f1004 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:23.693074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f2981-f969-484a-b2b6-1c5c3353c0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:23.693658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f2981-f969-484a-b2b6-1c5c3353c0cc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.693946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f2981-f969-484a-b2b6-1c5c3353c0cc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.694270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f2981-f969-484a-b2b6-1c5c3353c0cc None admin] PATH_INFO: `/v3/users/a8a2ecd8224a4650a78db92bfdadeacd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.710251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f2981-f969-484a-b2b6-1c5c3353c0cc None admin] RBAC: Authorizing `identity:get_user(user_id=a8a2ecd8224a4650a78db92bfdadeacd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:23.711727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f2981-f969-484a-b2b6-1c5c3353c0cc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:23.713842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 151/305] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:23 2022] GET /identity/v3/users/a8a2ecd8224a4650a78db92bfdadeacd => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:23.722881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f2981-f969-484a-b2b6-1c5c3353c0cc None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:23.727090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:23.787494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc517e3-cb99-49b8-8b5b-ce7dc57bde8e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:23.831036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c88016-199f-424b-8dae-c3878db1fda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:23.831708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c88016-199f-424b-8dae-c3878db1fda1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.831994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c88016-199f-424b-8dae-c3878db1fda1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.832271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c88016-199f-424b-8dae-c3878db1fda1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.844631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f2981-f969-484a-b2b6-1c5c3353c0cc None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:23.846034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c88016-199f-424b-8dae-c3878db1fda1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-c3c88016-199f-424b-8dae-c3878db1fda1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:23.852049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:23.852992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:23.857866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 155/306] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:23.859865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c88016-199f-424b-8dae-c3878db1fda1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:23.890992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c6968-04f8-4938-a507-56f4a414a6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:23.891597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c6968-04f8-4938-a507-56f4a414a6e4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.891895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c6968-04f8-4938-a507-56f4a414a6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.892152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c6968-04f8-4938-a507-56f4a414a6e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.892884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c6968-04f8-4938-a507-56f4a414a6e4 None admin] RBAC: Adding query filter params (role.id=935445ed062541ceb33d5021be2b092a, user.id=a8a2ecd8224a4650a78db92bfdadeacd, scope.system=all) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:23.893197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c6968-04f8-4938-a507-56f4a414a6e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:23.894685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c6968-04f8-4938-a507-56f4a414a6e4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:23.901575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 152/307] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Sep 13 19:35:23 2022] GET /identity/v3/role_assignments?user.id=a8a2ecd8224a4650a78db92bfdadeacd&scope.system=all&role.id=935445ed062541ceb33d5021be2b092a => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:23.906404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aff1c-f7a0-4546-a335-65e119b06989 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.906920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aff1c-f7a0-4546-a335-65e119b06989 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.907342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00aff1c-f7a0-4546-a335-65e119b06989 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.909212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 153/308] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:23.915451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeac83-ac02-4f97-8b25-4a6546255336 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:23.915981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeac83-ac02-4f97-8b25-4a6546255336 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:23.916256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeac83-ac02-4f97-8b25-4a6546255336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:23.939516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-22aeac83-ac02-4f97-8b25-4a6546255336 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:23.940384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:23.940978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-22aeac83-ac02-4f97-8b25-4a6546255336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:24.021895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c88016-199f-424b-8dae-c3878db1fda1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:24.037954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:24.050194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 154/309] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:24.073951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:24.076407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:24.077005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:24.077508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:24.078465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:24.079037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:24.080882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:24.092259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 156/310] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:24.100310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeac83-ac02-4f97-8b25-4a6546255336 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:24.212405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeac83-ac02-4f97-8b25-4a6546255336 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:24.252896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b40ac-d1f8-4e8e-af56-e70b1a9f559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:24.253967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b40ac-d1f8-4e8e-af56-e70b1a9f559c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:24.254448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b40ac-d1f8-4e8e-af56-e70b1a9f559c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:24.254943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b40ac-d1f8-4e8e-af56-e70b1a9f559c None admin] PATH_INFO: `/v3/users/b3fbbdf3d25544738102b678430f0c10` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:24.256353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b40ac-d1f8-4e8e-af56-e70b1a9f559c None admin] RBAC: Authorizing `identity:get_user(user_id=b3fbbdf3d25544738102b678430f0c10)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:24.258121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b40ac-d1f8-4e8e-af56-e70b1a9f559c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:24.260312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 155/311] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:24 2022] GET /identity/v3/users/b3fbbdf3d25544738102b678430f0c10 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:24.266775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:24.337890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b40ac-d1f8-4e8e-af56-e70b1a9f559c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:24.402012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6e591-c27f-438c-a075-04f4dff2eec0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:24.415474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:24.426047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:24.461754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb4b10-814c-4dd8-b301-d6636be819eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:24.462872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4b10-814c-4dd8-b301-d6636be819eb None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:24.463428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4b10-814c-4dd8-b301-d6636be819eb None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:24.463716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb4b10-814c-4dd8-b301-d6636be819eb None admin] PATH_INFO: `/v3/system/users/b3fbbdf3d25544738102b678430f0c10/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:24.465486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b40ac-d1f8-4e8e-af56-e70b1a9f559c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:24.465646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb4b10-814c-4dd8-b301-d6636be819eb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b3fbbdf3d25544738102b678430f0c10, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:24.466948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb4b10-814c-4dd8-b301-d6636be819eb None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:24.479005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 157/312] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Sep 13 19:35:24 2022] PUT /identity/v3/system/users/b3fbbdf3d25544738102b678430f0c10/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:24.514334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb28e5-d844-455e-a29b-23c13252afb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:24.515152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb28e5-d844-455e-a29b-23c13252afb3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:24.515433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb28e5-d844-455e-a29b-23c13252afb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:24.515704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb28e5-d844-455e-a29b-23c13252afb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:24.528782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb28e5-d844-455e-a29b-23c13252afb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-8adb28e5-d844-455e-a29b-23c13252afb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:24.534280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:24.535217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:24.536013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 156/313] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:24.541730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb4b10-814c-4dd8-b301-d6636be819eb None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:24.672714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb4b10-814c-4dd8-b301-d6636be819eb None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:24.715594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:24.716246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:24.716640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:24.717030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:24.717790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:24.718112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:24.719574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:24.733738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 158/314] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:24.735394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb28e5-d844-455e-a29b-23c13252afb3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:24.852845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb28e5-d844-455e-a29b-23c13252afb3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:24.878764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:24.892691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da055b06-f871-47f8-8510-ccff7d12f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:24.893706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055b06-f871-47f8-8510-ccff7d12f69f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:24.894191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055b06-f871-47f8-8510-ccff7d12f69f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:24.894694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da055b06-f871-47f8-8510-ccff7d12f69f None admin] PATH_INFO: `/v3/users/1da32f4a120b485e98dde2636d778741` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:24.896124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da055b06-f871-47f8-8510-ccff7d12f69f None admin] RBAC: Authorizing `identity:get_user(user_id=1da32f4a120b485e98dde2636d778741)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:24.897950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da055b06-f871-47f8-8510-ccff7d12f69f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:24.905122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 157/315] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:24 2022] GET /identity/v3/users/1da32f4a120b485e98dde2636d778741 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:24.906816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:24.950401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:25.032611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635fe6bf-eec9-4839-a548-06ffeabab517 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:25.069546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4f486-ae59-4393-812d-ef6b2b7c8ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:25.070214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f486-ae59-4393-812d-ef6b2b7c8ed0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:25.070504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f486-ae59-4393-812d-ef6b2b7c8ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:25.070779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f486-ae59-4393-812d-ef6b2b7c8ed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:25.150198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4f486-ae59-4393-812d-ef6b2b7c8ed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:25.151721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4f486-ae59-4393-812d-ef6b2b7c8ed0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:25.198352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:25.259440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 159/316] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:25.265704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da055b06-f871-47f8-8510-ccff7d12f69f None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:25.403666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da055b06-f871-47f8-8510-ccff7d12f69f None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:25.442305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb372fe-a8bd-4a9f-8cb8-d07736d60817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:25.443318 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb372fe-a8bd-4a9f-8cb8-d07736d60817 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:25.443795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb372fe-a8bd-4a9f-8cb8-d07736d60817 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:25.444300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb372fe-a8bd-4a9f-8cb8-d07736d60817 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:25.445685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb372fe-a8bd-4a9f-8cb8-d07736d60817 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:25.447594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb372fe-a8bd-4a9f-8cb8-d07736d60817 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:25.449946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 158/317] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:25 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:25.457669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4f486-ae59-4393-812d-ef6b2b7c8ed0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:25.564575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4f486-ae59-4393-812d-ef6b2b7c8ed0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:25.602164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497e86c-6e4e-40ea-8f1c-d8f6abeeb85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:25.602778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497e86c-6e4e-40ea-8f1c-d8f6abeeb85a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:25.603061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497e86c-6e4e-40ea-8f1c-d8f6abeeb85a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:25.603336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497e86c-6e4e-40ea-8f1c-d8f6abeeb85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:25.604076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497e86c-6e4e-40ea-8f1c-d8f6abeeb85a None admin] RBAC: Adding query filter params (role.id=935445ed062541ceb33d5021be2b092a, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc, user.id=1da32f4a120b485e98dde2636d778741) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:25.604389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497e86c-6e4e-40ea-8f1c-d8f6abeeb85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:25.605965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4497e86c-6e4e-40ea-8f1c-d8f6abeeb85a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:25.610871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:25.618037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 160/318] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:25 2022] GET /identity/v3/role_assignments?user.id=1da32f4a120b485e98dde2636d778741&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=935445ed062541ceb33d5021be2b092a => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:26.074985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05e27-2b8c-4040-b44c-779420ea2411 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.075393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05e27-2b8c-4040-b44c-779420ea2411 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.075523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a05e27-2b8c-4040-b44c-779420ea2411 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.077079 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 159/319] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:26.082551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7a1f-6b7d-4948-80a5-314dbd046a61 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.082840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7a1f-6b7d-4948-80a5-314dbd046a61 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.083118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7a1f-6b7d-4948-80a5-314dbd046a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.102455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-bf6d7a1f-6b7d-4948-80a5-314dbd046a61 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:26.103685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bf6d7a1f-6b7d-4948-80a5-314dbd046a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:26.189636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 161/320] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:26.336021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cd00f-ec3f-4cd8-9379-1a6aaa5e54d8 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.336536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cd00f-ec3f-4cd8-9379-1a6aaa5e54d8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.336536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cd00f-ec3f-4cd8-9379-1a6aaa5e54d8 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.338053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 160/321] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:26.344394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223bad0-bc8d-4996-9080-9edcbe2b0e2a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.344854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223bad0-bc8d-4996-9080-9edcbe2b0e2a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.345300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223bad0-bc8d-4996-9080-9edcbe2b0e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.374376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7223bad0-bc8d-4996-9080-9edcbe2b0e2a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:26.376251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7223bad0-bc8d-4996-9080-9edcbe2b0e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:26.453658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 162/322] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:26.456006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cd00f-ec3f-4cd8-9379-1a6aaa5e54d8 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:26.525188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:26.573892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cd00f-ec3f-4cd8-9379-1a6aaa5e54d8 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:26.619021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c240b-17fd-4454-96a6-3e289a79b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:26.619682 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c240b-17fd-4454-96a6-3e289a79b0fd None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.619965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c240b-17fd-4454-96a6-3e289a79b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.620239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c240b-17fd-4454-96a6-3e289a79b0fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.621977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c240b-17fd-4454-96a6-3e289a79b0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:26.623440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c240b-17fd-4454-96a6-3e289a79b0fd None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:26.625284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 161/323] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:26.630613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223bad0-bc8d-4996-9080-9edcbe2b0e2a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:26.699815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:26.737104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c240b-17fd-4454-96a6-3e289a79b0fd None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:26.760595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223bad0-bc8d-4996-9080-9edcbe2b0e2a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:26.800751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc8b21-0ad6-4a4d-b6a0-fcace5ad93c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:26.801817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc8b21-0ad6-4a4d-b6a0-fcace5ad93c0 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.802341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc8b21-0ad6-4a4d-b6a0-fcace5ad93c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.802678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc8b21-0ad6-4a4d-b6a0-fcace5ad93c0 None admin] PATH_INFO: `/v3/groups` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.803337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc8b21-0ad6-4a4d-b6a0-fcace5ad93c0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:26.805152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc8b21-0ad6-4a4d-b6a0-fcace5ad93c0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:26.821667 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 163/324] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Sep 13 19:35:26 2022] POST /identity/v3/groups => generated 226 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:26.845977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168b12-4842-44ad-ab3a-adcf10a4034b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.846315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168b12-4842-44ad-ab3a-adcf10a4034b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.846548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168b12-4842-44ad-ab3a-adcf10a4034b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.848138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 164/325] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:26.855084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c730f-6925-4737-8394-ea314355e382 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.855435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c730f-6925-4737-8394-ea314355e382 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.855680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c730f-6925-4737-8394-ea314355e382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.855842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c240b-17fd-4454-96a6-3e289a79b0fd None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:26.875802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-422c730f-6925-4737-8394-ea314355e382 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:26.876951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-422c730f-6925-4737-8394-ea314355e382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:26.890926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:26.901063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f45306-f714-421f-b8d5-5a1521ccc608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:26.901754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45306-f714-421f-b8d5-5a1521ccc608 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:26.902164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45306-f714-421f-b8d5-5a1521ccc608 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:26.903213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f45306-f714-421f-b8d5-5a1521ccc608 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:26.916036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f45306-f714-421f-b8d5-5a1521ccc608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-14f45306-f714-421f-b8d5-5a1521ccc608 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:26.925754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:26.926727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:26.927534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 162/326] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:26.930172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f45306-f714-421f-b8d5-5a1521ccc608 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:26.974220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 165/327] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:27.065424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f45306-f714-421f-b8d5-5a1521ccc608 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:27.104670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:27.105377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:27.106191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:27.106465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:27.107155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:27.107460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:27.108904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:27.119056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 163/328] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:27.125497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c730f-6925-4737-8394-ea314355e382 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:27.193940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:27.241522 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:27.270889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c730f-6925-4737-8394-ea314355e382 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:27.321635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:27.325027 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f313d3-f54e-49f7-97ba-acc51c4b71c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:27.326769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f313d3-f54e-49f7-97ba-acc51c4b71c6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:27.326769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f313d3-f54e-49f7-97ba-acc51c4b71c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:27.326971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f313d3-f54e-49f7-97ba-acc51c4b71c6 None admin] PATH_INFO: `/v3/users/a8a2ecd8224a4650a78db92bfdadeacd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:27.330424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f313d3-f54e-49f7-97ba-acc51c4b71c6 None admin] RBAC: Authorizing `identity:get_user(user_id=a8a2ecd8224a4650a78db92bfdadeacd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:27.332535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f313d3-f54e-49f7-97ba-acc51c4b71c6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:27.336135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 166/329] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:27 2022] GET /identity/v3/users/a8a2ecd8224a4650a78db92bfdadeacd => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:27.340683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f313d3-f54e-49f7-97ba-acc51c4b71c6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:27.399589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67037d-c144-412c-9b2f-43f041cce6a9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:27.443621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b5124-6b28-4eb7-8587-51d98190d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:27.445479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b5124-6b28-4eb7-8587-51d98190d768 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:27.445949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b5124-6b28-4eb7-8587-51d98190d768 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:27.446364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b5124-6b28-4eb7-8587-51d98190d768 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:27.460763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b5124-6b28-4eb7-8587-51d98190d768 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-721b5124-6b28-4eb7-8587-51d98190d768 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:27.468087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:27.469113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:27.470156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:27.472870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 164/330] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:27.481074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b5124-6b28-4eb7-8587-51d98190d768 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:27.497897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f313d3-f54e-49f7-97ba-acc51c4b71c6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:27.532585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:27.562541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c9e9c-4e99-4d13-8171-912195c5f25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:27.563215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c9e9c-4e99-4d13-8171-912195c5f25e None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:27.563495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c9e9c-4e99-4d13-8171-912195c5f25e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:27.563769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c9e9c-4e99-4d13-8171-912195c5f25e None admin] PATH_INFO: `/v3/system/users/a8a2ecd8224a4650a78db92bfdadeacd/roles/935445ed062541ceb33d5021be2b092a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:27.566449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c9e9c-4e99-4d13-8171-912195c5f25e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a8a2ecd8224a4650a78db92bfdadeacd, role_id=935445ed062541ceb33d5021be2b092a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:27.567918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c9e9c-4e99-4d13-8171-912195c5f25e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:27.578429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 167/331] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Sep 13 19:35:27 2022] PUT /identity/v3/system/users/a8a2ecd8224a4650a78db92bfdadeacd/roles/935445ed062541ceb33d5021be2b092a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:27.635298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b5124-6b28-4eb7-8587-51d98190d768 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:27.654901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:27.686670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:27.687360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:27.687680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:27.687968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:27.688762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:27.689072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:27.690578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:27.702364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 165/332] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:27.708320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c9e9c-4e99-4d13-8171-912195c5f25e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:27.853692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c9e9c-4e99-4d13-8171-912195c5f25e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:27.897246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d078271-8f97-48f4-8445-0c84daea7438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:27.897925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d078271-8f97-48f4-8445-0c84daea7438 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:27.898208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d078271-8f97-48f4-8445-0c84daea7438 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:27.898476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d078271-8f97-48f4-8445-0c84daea7438 None admin] PATH_INFO: `/v3/users/b3fbbdf3d25544738102b678430f0c10` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:27.899656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d078271-8f97-48f4-8445-0c84daea7438 None admin] RBAC: Authorizing `identity:get_user(user_id=b3fbbdf3d25544738102b678430f0c10)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:27.901106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d078271-8f97-48f4-8445-0c84daea7438 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:27.903616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 168/333] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:27 2022] GET /identity/v3/users/b3fbbdf3d25544738102b678430f0c10 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:27.909161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:28.044643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff826aff-65d5-406c-a3d6-62ccfcb4405a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:28.082938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3adb7fc-beb7-4594-9da8-6dc920efe7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:28.083568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adb7fc-beb7-4594-9da8-6dc920efe7cc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:28.083847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adb7fc-beb7-4594-9da8-6dc920efe7cc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:28.084119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3adb7fc-beb7-4594-9da8-6dc920efe7cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:28.084852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adb7fc-beb7-4594-9da8-6dc920efe7cc None admin] RBAC: Adding query filter params (role.id=afc24e28d6c34708ab897fc4b7029bb2, user.id=b3fbbdf3d25544738102b678430f0c10, scope.system=all) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:28.085162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adb7fc-beb7-4594-9da8-6dc920efe7cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:28.086648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3adb7fc-beb7-4594-9da8-6dc920efe7cc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:28.100577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 166/334] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Sep 13 19:35:27 2022] GET /identity/v3/role_assignments?user.id=b3fbbdf3d25544738102b678430f0c10&scope.system=all&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:30.298823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3253ab-834d-48fe-a4e8-78a3d666ea06 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:30.299807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3253ab-834d-48fe-a4e8-78a3d666ea06 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:30.300296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3253ab-834d-48fe-a4e8-78a3d666ea06 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:30.302193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 169/335] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:30.308225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d57d9-1f3f-4860-ae10-5ceddc084dce None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:30.309639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d57d9-1f3f-4860-ae10-5ceddc084dce None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:30.310132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89d57d9-1f3f-4860-ae10-5ceddc084dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:30.330358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-c89d57d9-1f3f-4860-ae10-5ceddc084dce None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:30.331918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c89d57d9-1f3f-4860-ae10-5ceddc084dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:30.362354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:30.417393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 167/336] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:30.504457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c3b6-04a5-47cd-b686-b5f246fccefb None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:30.504772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c3b6-04a5-47cd-b686-b5f246fccefb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:30.505017 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032c3b6-04a5-47cd-b686-b5f246fccefb None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:30.506962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 170/337] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:30.514011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95a858-ebe9-4d53-b39b-53f6b6281428 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:30.514309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95a858-ebe9-4d53-b39b-53f6b6281428 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:30.514568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95a858-ebe9-4d53-b39b-53f6b6281428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:30.539233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4c95a858-ebe9-4d53-b39b-53f6b6281428 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:30.540451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4c95a858-ebe9-4d53-b39b-53f6b6281428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:30.584702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09808-0c5c-4888-94f8-d2d073fa4945 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:30.584990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09808-0c5c-4888-94f8-d2d073fa4945 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:30.585214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b09808-0c5c-4888-94f8-d2d073fa4945 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:30.587612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 171/338] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:30.594030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b77882-8d66-43fc-ae2f-9ec38c4177ce None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:30.594860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b77882-8d66-43fc-ae2f-9ec38c4177ce None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:30.595531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b77882-8d66-43fc-ae2f-9ec38c4177ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:30.644713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-87b77882-8d66-43fc-ae2f-9ec38c4177ce None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:30.649101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-87b77882-8d66-43fc-ae2f-9ec38c4177ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:30.691665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 168/339] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:30.734676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:30.761511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 172/340] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:30.787907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084b74a-de2b-4a8a-a4c3-680fbba4ef54 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:30.788517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084b74a-de2b-4a8a-a4c3-680fbba4ef54 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:30.788970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084b74a-de2b-4a8a-a4c3-680fbba4ef54 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:30.791216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 169/341] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:30.796910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcb97d-3bdd-4d72-bb1a-3702d40cada7 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:30.797495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcb97d-3bdd-4d72-bb1a-3702d40cada7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:30.797962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcb97d-3bdd-4d72-bb1a-3702d40cada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:30.826729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-45bcb97d-3bdd-4d72-bb1a-3702d40cada7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:30.828310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-45bcb97d-3bdd-4d72-bb1a-3702d40cada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:30.914658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 173/342] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:30.948208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084b74a-de2b-4a8a-a4c3-680fbba4ef54 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:31.074533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:31.087226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084b74a-de2b-4a8a-a4c3-680fbba4ef54 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:31.135722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3bb54-ad4b-4f8b-98df-adca519e023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:31.137122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bb54-ad4b-4f8b-98df-adca519e023d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:31.137409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bb54-ad4b-4f8b-98df-adca519e023d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:31.137710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3bb54-ad4b-4f8b-98df-adca519e023d None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:31.156798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3bb54-ad4b-4f8b-98df-adca519e023d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-e4c3bb54-ad4b-4f8b-98df-adca519e023d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:31.161968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:31.162843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:31.164100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 170/343] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:31.175199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcb97d-3bdd-4d72-bb1a-3702d40cada7 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:31.306300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3bb54-ad4b-4f8b-98df-adca519e023d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:31.325185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcb97d-3bdd-4d72-bb1a-3702d40cada7 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:31.340210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:31.401550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:31.402298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:31.402579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:31.402847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:31.403542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:31.403839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:31.405240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:31.420518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 174/344] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:31.421546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:31.491624 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3bb54-ad4b-4f8b-98df-adca519e023d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:31.529682 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf8130-3e60-48ba-832b-d2efce1f9dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:31.530227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8130-3e60-48ba-832b-d2efce1f9dd3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:31.530504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8130-3e60-48ba-832b-d2efce1f9dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:31.530779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bf8130-3e60-48ba-832b-d2efce1f9dd3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:31.544058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bf8130-3e60-48ba-832b-d2efce1f9dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-f5bf8130-3e60-48ba-832b-d2efce1f9dd3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:31.549172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:31.550296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:31.551146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 171/345] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:31.551872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf8130-3e60-48ba-832b-d2efce1f9dd3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:31.568651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:31.582377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48e534-4057-4fb7-bce8-08d4f04c0933 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:31.633249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc364a9a-e857-43ab-a5aa-b7e73bbf1aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:31.633829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc364a9a-e857-43ab-a5aa-b7e73bbf1aa7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:31.634111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc364a9a-e857-43ab-a5aa-b7e73bbf1aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:31.634385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc364a9a-e857-43ab-a5aa-b7e73bbf1aa7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:31.655815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc364a9a-e857-43ab-a5aa-b7e73bbf1aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-dc364a9a-e857-43ab-a5aa-b7e73bbf1aa7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:31.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:31.662583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:31.663510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 175/346] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:31.664214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc364a9a-e857-43ab-a5aa-b7e73bbf1aa7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:31.696953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bf8130-3e60-48ba-832b-d2efce1f9dd3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:31.713484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:31.755938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b172fe7-4509-462d-9ea0-ba6027f3aa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:31.757158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b172fe7-4509-462d-9ea0-ba6027f3aa60 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:31.760856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b172fe7-4509-462d-9ea0-ba6027f3aa60 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:31.760856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b172fe7-4509-462d-9ea0-ba6027f3aa60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:31.780449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b172fe7-4509-462d-9ea0-ba6027f3aa60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-4b172fe7-4509-462d-9ea0-ba6027f3aa60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:31.785769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:31.787022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:31.787835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:31.791499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 172/347] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:31.796733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b172fe7-4509-462d-9ea0-ba6027f3aa60 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:31.825239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc364a9a-e857-43ab-a5aa-b7e73bbf1aa7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:31.872371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b93ba-de82-46d7-8673-fe03895494f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:31.873529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b93ba-de82-46d7-8673-fe03895494f7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:31.873877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b93ba-de82-46d7-8673-fe03895494f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:31.874441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b93ba-de82-46d7-8673-fe03895494f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:31.891487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:31.943333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b172fe7-4509-462d-9ea0-ba6027f3aa60 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:31.975764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b93ba-de82-46d7-8673-fe03895494f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:31.977368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b93ba-de82-46d7-8673-fe03895494f7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:31.990987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:31.992006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:31.992437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:31.992840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:31.993534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:31.993891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:31.995270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:32.006864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 173/348] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:32.008326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:32.020961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:32.129730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 176/349] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:32.129730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b345f5a-7931-46e3-8ae5-1b82bc7135b3 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:32.129730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b345f5a-7931-46e3-8ae5-1b82bc7135b3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:32.129730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b345f5a-7931-46e3-8ae5-1b82bc7135b3 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:32.134851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 177/350] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:32.135765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b345f5a-7931-46e3-8ae5-1b82bc7135b3 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:32.180768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0da7a-1c3e-4972-9b7b-e0f963b79562 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:32.223413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:32.229684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:32.230734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:32.231333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:32.231582 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:32.232281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:32.232592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:32.234416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:32.248394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 174/351] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:32.249714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:32.278258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b345f5a-7931-46e3-8ae5-1b82bc7135b3 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:32.318551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:32.325497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:32.326244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:32.326585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:32.326692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:32.327356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:32.327655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:32.329970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:32.339451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 178/352] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:32.340516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:32.404576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d19d0-824b-4031-b88c-7df83fb5325d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:32.445745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8614-4e96-4c43-9c9a-7d2615d40a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:32.446390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8614-4e96-4c43-9c9a-7d2615d40a8e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:32.446929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8614-4e96-4c43-9c9a-7d2615d40a8e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:32.447199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8614-4e96-4c43-9c9a-7d2615d40a8e None admin] PATH_INFO: `/v3/users/a8a2ecd8224a4650a78db92bfdadeacd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:32.448376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8614-4e96-4c43-9c9a-7d2615d40a8e None admin] RBAC: Authorizing `identity:get_user(user_id=a8a2ecd8224a4650a78db92bfdadeacd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:32.450255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e8614-4e96-4c43-9c9a-7d2615d40a8e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:32.452347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 175/353] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:32 2022] GET /identity/v3/users/a8a2ecd8224a4650a78db92bfdadeacd => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:32.453809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d6f1d-dde0-4b7d-887d-44d03bfcb8d3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:32.454095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d6f1d-dde0-4b7d-887d-44d03bfcb8d3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:32.454365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d6f1d-dde0-4b7d-887d-44d03bfcb8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:32.475430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bc1d6f1d-dde0-4b7d-887d-44d03bfcb8d3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:32.476645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bc1d6f1d-dde0-4b7d-887d-44d03bfcb8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:32.515732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b0824-d803-4fdb-bb19-f0cfe6dc27a7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:32.551622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:32.559333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009f2df-ec18-4d13-9a9d-aa8d51ba7765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:32.559998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009f2df-ec18-4d13-9a9d-aa8d51ba7765 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:32.560280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009f2df-ec18-4d13-9a9d-aa8d51ba7765 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:32.560556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009f2df-ec18-4d13-9a9d-aa8d51ba7765 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:32.561859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009f2df-ec18-4d13-9a9d-aa8d51ba7765 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:32.563353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009f2df-ec18-4d13-9a9d-aa8d51ba7765 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:32.565244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 179/354] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:32 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:32.565914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009f2df-ec18-4d13-9a9d-aa8d51ba7765 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:32.566969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 176/355] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:32.568269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d6f1d-dde0-4b7d-887d-44d03bfcb8d3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:32.656152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:32.709892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009f2df-ec18-4d13-9a9d-aa8d51ba7765 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:32.712907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d6f1d-dde0-4b7d-887d-44d03bfcb8d3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:32.754984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d853e8-2ce4-4d92-9004-1066976770a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:32.755769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d853e8-2ce4-4d92-9004-1066976770a4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:32.756069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d853e8-2ce4-4d92-9004-1066976770a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:32.756336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d853e8-2ce4-4d92-9004-1066976770a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:32.759633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f9f6f-29ce-4815-934e-e80e389bf44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:32.760243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9f6f-29ce-4815-934e-e80e389bf44f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:32.760538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9f6f-29ce-4815-934e-e80e389bf44f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:32.760814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9f6f-29ce-4815-934e-e80e389bf44f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:32.854631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:32.855702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d853e8-2ce4-4d92-9004-1066976770a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:32.857334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d853e8-2ce4-4d92-9004-1066976770a4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:32.867252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f9f6f-29ce-4815-934e-e80e389bf44f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:32.871856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f9f6f-29ce-4815-934e-e80e389bf44f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:32.949403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:33.004650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 180/356] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:33.005874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d853e8-2ce4-4d92-9004-1066976770a4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.028155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 177/357] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:33.028377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f143557-63b8-4c1a-9dd6-358feb3805ee None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.029016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f143557-63b8-4c1a-9dd6-358feb3805ee None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.029016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f143557-63b8-4c1a-9dd6-358feb3805ee None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.031553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 178/358] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:33.032134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f143557-63b8-4c1a-9dd6-358feb3805ee None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.139665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d853e8-2ce4-4d92-9004-1066976770a4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:33.170643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:33.180509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:33.181128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.181406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.182420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.183729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] RBAC: Adding query filter params (role.id=935445ed062541ceb33d5021be2b092a, user.id=a8a2ecd8224a4650a78db92bfdadeacd, scope.system=all) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:33.184284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:33.185863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:33.188556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f143557-63b8-4c1a-9dd6-358feb3805ee None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:33.204071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 181/359] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/role_assignments?user.id=a8a2ecd8224a4650a78db92bfdadeacd&scope.system=all&role.id=935445ed062541ceb33d5021be2b092a => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:33.204862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.231531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:33.246862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3bed8-7a70-4ed5-99ca-a2236bee2d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:33.247952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3bed8-7a70-4ed5-99ca-a2236bee2d9b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.248446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3bed8-7a70-4ed5-99ca-a2236bee2d9b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.248813 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3bed8-7a70-4ed5-99ca-a2236bee2d9b None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.260290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3bed8-7a70-4ed5-99ca-a2236bee2d9b None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:33.262806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b3bed8-7a70-4ed5-99ca-a2236bee2d9b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:33.270212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 179/360] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:33.271906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3bed8-7a70-4ed5-99ca-a2236bee2d9b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.348263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc14ee7-6c51-4f42-ad58-b5203c8a1267 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:33.403327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec24840-f91f-4543-9037-1eec3e6e6c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:33.403691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec24840-f91f-4543-9037-1eec3e6e6c82 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.403896 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec24840-f91f-4543-9037-1eec3e6e6c82 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.404988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec24840-f91f-4543-9037-1eec3e6e6c82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.422123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec24840-f91f-4543-9037-1eec3e6e6c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-9ec24840-f91f-4543-9037-1eec3e6e6c82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:33.427702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:33.428659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:33.429462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:33.432311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3bed8-7a70-4ed5-99ca-a2236bee2d9b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:33.433756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 182/361] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:33.435162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec24840-f91f-4543-9037-1eec3e6e6c82 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.473708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea47ee36-cf79-4a8d-84f2-8caefe197b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:33.474925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47ee36-cf79-4a8d-84f2-8caefe197b5c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.475865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47ee36-cf79-4a8d-84f2-8caefe197b5c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.476321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea47ee36-cf79-4a8d-84f2-8caefe197b5c None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.477566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47ee36-cf79-4a8d-84f2-8caefe197b5c None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:33.479093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea47ee36-cf79-4a8d-84f2-8caefe197b5c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:33.481502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 180/362] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:33.484042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999898-cb7c-42d2-a265-02d0c098ab32 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.484042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999898-cb7c-42d2-a265-02d0c098ab32 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.484042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999898-cb7c-42d2-a265-02d0c098ab32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.506593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9f999898-cb7c-42d2-a265-02d0c098ab32 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:33.511708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9f999898-cb7c-42d2-a265-02d0c098ab32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:33.530860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:33.554098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:33.582943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec24840-f91f-4543-9037-1eec3e6e6c82 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:33.609583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 181/363] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:33.611482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f999898-cb7c-42d2-a265-02d0c098ab32 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.639836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072df0f-b36e-4037-8834-73a7b3d9359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:33.640488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072df0f-b36e-4037-8834-73a7b3d9359e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.640771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072df0f-b36e-4037-8834-73a7b3d9359e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.641048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072df0f-b36e-4037-8834-73a7b3d9359e None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.643226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072df0f-b36e-4037-8834-73a7b3d9359e None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:33.644679 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072df0f-b36e-4037-8834-73a7b3d9359e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:33.651162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 183/364] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:33.651162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072df0f-b36e-4037-8834-73a7b3d9359e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.764992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f999898-cb7c-42d2-a265-02d0c098ab32 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:33.790304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072df0f-b36e-4037-8834-73a7b3d9359e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:33.822049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:33.822671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.822947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.823222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.824560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] RBAC: Adding query filter params (group.id=dc0d5de3eb9945d788fd23bad20d05a9, role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:33.824869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:33.828593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:33.833164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 182/365] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/role_assignments?group.id=dc0d5de3eb9945d788fd23bad20d05a9&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:33.838657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.853084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:33.854060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:33.854399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:33.854683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:33.855993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:33.856387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:33.858128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:33.875523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 184/366] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:33.876343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:33.880372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:33.907683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:34.019394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1571fd-928c-42cc-a006-d737fb4ac000 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:34.050683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3481f88-4634-4e5e-8699-96b614cd000e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:34.084504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199702a-ef31-41f9-bf67-04552e06aa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:34.085157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199702a-ef31-41f9-bf67-04552e06aa4e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:34.085433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199702a-ef31-41f9-bf67-04552e06aa4e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:34.086030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199702a-ef31-41f9-bf67-04552e06aa4e None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:34.087053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199702a-ef31-41f9-bf67-04552e06aa4e None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:34.088963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199702a-ef31-41f9-bf67-04552e06aa4e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:34.091506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 183/367] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:34.092551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199702a-ef31-41f9-bf67-04552e06aa4e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:34.109363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed38a8-a52c-4830-bef7-35c129ce95a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:34.110064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed38a8-a52c-4830-bef7-35c129ce95a6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:34.110351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed38a8-a52c-4830-bef7-35c129ce95a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:34.110628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed38a8-a52c-4830-bef7-35c129ce95a6 None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:34.111800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed38a8-a52c-4830-bef7-35c129ce95a6 None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:34.113280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed38a8-a52c-4830-bef7-35c129ce95a6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:34.114953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 185/368] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:33 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:34.115719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed38a8-a52c-4830-bef7-35c129ce95a6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:34.215002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:34.219179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:34.238718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199702a-ef31-41f9-bf67-04552e06aa4e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:34.241088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed38a8-a52c-4830-bef7-35c129ce95a6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:34.284201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0db159-a59a-4cad-ad4f-40bcee8ffb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:34.284738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0db159-a59a-4cad-ad4f-40bcee8ffb54 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:34.285019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0db159-a59a-4cad-ad4f-40bcee8ffb54 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:34.285360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0db159-a59a-4cad-ad4f-40bcee8ffb54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:34.287461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04690c7-b634-4c8b-bc54-cb24e1e4c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:34.288073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04690c7-b634-4c8b-bc54-cb24e1e4c84c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:34.288357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04690c7-b634-4c8b-bc54-cb24e1e4c84c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:34.288629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04690c7-b634-4c8b-bc54-cb24e1e4c84c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:34.298618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0db159-a59a-4cad-ad4f-40bcee8ffb54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-7a0db159-a59a-4cad-ad4f-40bcee8ffb54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:34.305715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:34.306685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:34.307530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 186/369] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:34.308215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0db159-a59a-4cad-ad4f-40bcee8ffb54 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:34.374011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04690c7-b634-4c8b-bc54-cb24e1e4c84c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:34.375495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04690c7-b634-4c8b-bc54-cb24e1e4c84c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:34.434807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0db159-a59a-4cad-ad4f-40bcee8ffb54 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:34.473568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:34.474687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:34.475102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:34.475598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:34.476965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] RBAC: Adding query filter params (group.id=dc0d5de3eb9945d788fd23bad20d05a9, role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:34.477075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:34.477765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:34.479038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:34.485252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 187/370] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:34 2022] GET /identity/v3/role_assignments?group.id=dc0d5de3eb9945d788fd23bad20d05a9&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=90097e8ce1394daba58b200de808ad59 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:34.486119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:34.489963 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:34.507382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 184/371] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:34.508491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04690c7-b634-4c8b-bc54-cb24e1e4c84c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:34.623858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab37c93-67b8-47be-9d3a-9ef0ad47220f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:34.656072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04690c7-b634-4c8b-bc54-cb24e1e4c84c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:34.665330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:34.666221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:34.666635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:34.667012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:34.667949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] RBAC: Adding query filter params (group.id=dc0d5de3eb9945d788fd23bad20d05a9, role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:34.668451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:34.670257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:34.677443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 188/372] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:34 2022] GET /identity/v3/role_assignments?group.id=dc0d5de3eb9945d788fd23bad20d05a9&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:34.678505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:34.726083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:34.726740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:34.727018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:34.727289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:34.727975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:34.728277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:34.729676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:34.745279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 185/373] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:34.753377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:34.773730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:34.838743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e9754-af94-430f-8d60-7d5af613543b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:34.860073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:34.889542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd087d1-fcc1-4a9d-a61c-2d931b4c3f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:34.891143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd087d1-fcc1-4a9d-a61c-2d931b4c3f22 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:34.891914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd087d1-fcc1-4a9d-a61c-2d931b4c3f22 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:34.892650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd087d1-fcc1-4a9d-a61c-2d931b4c3f22 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:34.895211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd087d1-fcc1-4a9d-a61c-2d931b4c3f22 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:34.898358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd087d1-fcc1-4a9d-a61c-2d931b4c3f22 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:34.902181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 189/374] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:34 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:34.908727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd087d1-fcc1-4a9d-a61c-2d931b4c3f22 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:34.999550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c6a67-f5b8-43b9-a5fb-940a73a61ead None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:35.040628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95a80b-c1c2-41c7-925f-6e1f56ba3588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:35.041263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a80b-c1c2-41c7-925f-6e1f56ba3588 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:35.041543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a80b-c1c2-41c7-925f-6e1f56ba3588 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:35.043728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95a80b-c1c2-41c7-925f-6e1f56ba3588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:35.111338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd087d1-fcc1-4a9d-a61c-2d931b4c3f22 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:35.163846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d896515-75b1-470a-9ee4-06b7771fb8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:35.164517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d896515-75b1-470a-9ee4-06b7771fb8d9 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:35.164806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d896515-75b1-470a-9ee4-06b7771fb8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:35.165088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d896515-75b1-470a-9ee4-06b7771fb8d9 None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:35.168082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d896515-75b1-470a-9ee4-06b7771fb8d9 None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:35.169734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d896515-75b1-470a-9ee4-06b7771fb8d9 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:35.171805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 190/375] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:34 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:35.177789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d896515-75b1-470a-9ee4-06b7771fb8d9 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:35.182662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:35.184195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95a80b-c1c2-41c7-925f-6e1f56ba3588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:35.185721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95a80b-c1c2-41c7-925f-6e1f56ba3588 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:35.238122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:35.293236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 186/376] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:35:35.302690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95a80b-c1c2-41c7-925f-6e1f56ba3588 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:35.411612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d896515-75b1-470a-9ee4-06b7771fb8d9 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:35.420723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95a80b-c1c2-41c7-925f-6e1f56ba3588 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:35.462024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bac70-d76b-4b06-8096-a95a5e74d8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:35.462673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bac70-d76b-4b06-8096-a95a5e74d8df None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:35.462951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bac70-d76b-4b06-8096-a95a5e74d8df None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:35.463219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bac70-d76b-4b06-8096-a95a5e74d8df None admin] PATH_INFO: `/v3/projects/61d39e602be54c45a9fb48df499e3bc7` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:35.465714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bac70-d76b-4b06-8096-a95a5e74d8df None admin] RBAC: Authorizing `identity:get_project(project_id=61d39e602be54c45a9fb48df499e3bc7)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:35.467164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bac70-d76b-4b06-8096-a95a5e74d8df None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:35.468840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 187/377] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:35 2022] GET /identity/v3/projects/61d39e602be54c45a9fb48df499e3bc7 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:35.478866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bac70-d76b-4b06-8096-a95a5e74d8df None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:35.483877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:35.485093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:35.485574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:35.486064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:35.487083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] RBAC: Adding query filter params (group.id=dc0d5de3eb9945d788fd23bad20d05a9, role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:35.487578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:35.489214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:35.496345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 191/378] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:35 2022] GET /identity/v3/role_assignments?group.id=dc0d5de3eb9945d788fd23bad20d05a9&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=90097e8ce1394daba58b200de808ad59 => generated 260 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:35.514003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:35.648681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bac70-d76b-4b06-8096-a95a5e74d8df None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:35.706543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da179f56-c43d-4535-bad4-ce2b5d365e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:35.707175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179f56-c43d-4535-bad4-ce2b5d365e5b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:35.707441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179f56-c43d-4535-bad4-ce2b5d365e5b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:35.707661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da179f56-c43d-4535-bad4-ce2b5d365e5b None admin] PATH_INFO: `/v3/groups/0df7c14ac803403399c7d9ace881a4dc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:35.713402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da179f56-c43d-4535-bad4-ce2b5d365e5b None admin] RBAC: Authorizing `identity:get_group(group_id=0df7c14ac803403399c7d9ace881a4dc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:35.715474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da179f56-c43d-4535-bad4-ce2b5d365e5b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:35.717548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 188/379] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:35 2022] GET /identity/v3/groups/0df7c14ac803403399c7d9ace881a4dc => generated 229 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:35.723716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:35.841093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:35.905291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ea142-3765-44af-8997-3c3f845ae15b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:35.955721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac120c-2342-4541-b218-544d8e55ed28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:35.956466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac120c-2342-4541-b218-544d8e55ed28 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:35.956825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac120c-2342-4541-b218-544d8e55ed28 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:35.957103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac120c-2342-4541-b218-544d8e55ed28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:35.957957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac120c-2342-4541-b218-544d8e55ed28 None admin] RBAC: Adding query filter params (group.id=0df7c14ac803403399c7d9ace881a4dc, role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=61d39e602be54c45a9fb48df499e3bc7) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:35.958378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac120c-2342-4541-b218-544d8e55ed28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:35.959936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac120c-2342-4541-b218-544d8e55ed28 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:35.970518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 192/380] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:35 2022] GET /identity/v3/role_assignments?group.id=0df7c14ac803403399c7d9ace881a4dc&scope.project.id=61d39e602be54c45a9fb48df499e3bc7&role.id=b698800ceec3460a8519ea627650da0f => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:36.340330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398754e2-a951-4627-ae5d-e533d9d79467 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:36.341003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398754e2-a951-4627-ae5d-e533d9d79467 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:36.341479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398754e2-a951-4627-ae5d-e533d9d79467 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:36.343336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 189/381] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:36.349087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ea81e-8040-4d66-93ab-80eace725442 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:36.349693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ea81e-8040-4d66-93ab-80eace725442 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:36.350163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ea81e-8040-4d66-93ab-80eace725442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:36.370231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-138ea81e-8040-4d66-93ab-80eace725442 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:36.371868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-138ea81e-8040-4d66-93ab-80eace725442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:36.443895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 193/382] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:36.905060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398754e2-a951-4627-ae5d-e533d9d79467 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:37.025979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:37.043348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398754e2-a951-4627-ae5d-e533d9d79467 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:37.097523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d632417-1330-4c7f-8525-e0281cf8acac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:37.098505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632417-1330-4c7f-8525-e0281cf8acac None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:37.098930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632417-1330-4c7f-8525-e0281cf8acac None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:37.099351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d632417-1330-4c7f-8525-e0281cf8acac None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:37.125493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d632417-1330-4c7f-8525-e0281cf8acac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-4d632417-1330-4c7f-8525-e0281cf8acac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:37.134062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:37.134985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:37.135775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:37.141766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 190/383] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:37.146749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ea81e-8040-4d66-93ab-80eace725442 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:37.296551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:37.398368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ea81e-8040-4d66-93ab-80eace725442 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:37.432261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759290e-2e6b-4523-b485-97ec070ea9cd None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:37.432580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759290e-2e6b-4523-b485-97ec070ea9cd None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:37.432831 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759290e-2e6b-4523-b485-97ec070ea9cd None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:37.436437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 191/384] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:37.442642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:37.443264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:37.443548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:37.443823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:37.444493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:37.444799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:37.446218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:37.451631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6617-fd73-412b-9857-1c7a51a9c611 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:37.452089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6617-fd73-412b-9857-1c7a51a9c611 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:37.452542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be6617-fd73-412b-9857-1c7a51a9c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:37.470929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 194/385] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:37.476142 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:37.487203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-50be6617-fd73-412b-9857-1c7a51a9c611 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:37.490390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-50be6617-fd73-412b-9857-1c7a51a9c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:37.634769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 192/386] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:37.661119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf75b45-90d0-4ea9-bf82-fb3dfce4125d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:37.696066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:37.718355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d9a352-ed65-4b42-98c6-6452e5142014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:37.719016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9a352-ed65-4b42-98c6-6452e5142014 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:37.719305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9a352-ed65-4b42-98c6-6452e5142014 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:37.719588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d9a352-ed65-4b42-98c6-6452e5142014 None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:37.720770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9a352-ed65-4b42-98c6-6452e5142014 None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:37.722258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d9a352-ed65-4b42-98c6-6452e5142014 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:37.724266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 195/387] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:37 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:37.729955 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be6617-fd73-412b-9857-1c7a51a9c611 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:37.831401 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:37.874747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0283c-3bac-444c-9a22-937cc4b902f1 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:37.875071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0283c-3bac-444c-9a22-937cc4b902f1 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:37.875309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0283c-3bac-444c-9a22-937cc4b902f1 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:37.876733 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 196/388] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:37.879647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be6617-fd73-412b-9857-1c7a51a9c611 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:37.896138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5454b-61ba-4f97-a58c-6da853b00208 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:37.896409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5454b-61ba-4f97-a58c-6da853b00208 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:37.896603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5454b-61ba-4f97-a58c-6da853b00208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:37.926030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-cfe5454b-61ba-4f97-a58c-6da853b00208 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:37.927504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cfe5454b-61ba-4f97-a58c-6da853b00208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:37.953717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3591ba-c392-4bef-8a4a-8e6e607fc49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:37.954425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3591ba-c392-4bef-8a4a-8e6e607fc49c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:37.954648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3591ba-c392-4bef-8a4a-8e6e607fc49c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:37.954925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3591ba-c392-4bef-8a4a-8e6e607fc49c None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:37.958068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3591ba-c392-4bef-8a4a-8e6e607fc49c None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:37.959800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3591ba-c392-4bef-8a4a-8e6e607fc49c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:37.961658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 193/389] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:37 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:37.970817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3591ba-c392-4bef-8a4a-8e6e607fc49c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.039245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 197/390] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:38.040120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5454b-61ba-4f97-a58c-6da853b00208 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.159169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3591ba-c392-4bef-8a4a-8e6e607fc49c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:38.173905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5454b-61ba-4f97-a58c-6da853b00208 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:38.196240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:38.216118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73176779-981f-4abf-8469-af7bad9c73fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:38.216118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73176779-981f-4abf-8469-af7bad9c73fe None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.216118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73176779-981f-4abf-8469-af7bad9c73fe None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.216118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73176779-981f-4abf-8469-af7bad9c73fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.228401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73176779-981f-4abf-8469-af7bad9c73fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:38.230131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4a932-a123-40e6-b783-b686b2656925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:38.231081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4a932-a123-40e6-b783-b686b2656925 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.231511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4a932-a123-40e6-b783-b686b2656925 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.231981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba4a932-a123-40e6-b783-b686b2656925 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/groups/dc0d5de3eb9945d788fd23bad20d05a9/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-73176779-981f-4abf-8469-af7bad9c73fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:38.234959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:38.236149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:38.237162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:38.241898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 198/391] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:38.242540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba4a932-a123-40e6-b783-b686b2656925 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9b796cfb79246dca28d73123bc5e6bc, group_id=dc0d5de3eb9945d788fd23bad20d05a9, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:38.244083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba4a932-a123-40e6-b783-b686b2656925 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:38.248056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73176779-981f-4abf-8469-af7bad9c73fe None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.264642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:38.274334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 194/392] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 13 19:35:37 2022] PUT /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/groups/dc0d5de3eb9945d788fd23bad20d05a9/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:38.385377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73176779-981f-4abf-8469-af7bad9c73fe None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:38.408027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4a932-a123-40e6-b783-b686b2656925 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.443533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:38.446417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.448885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.449655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.451329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:38.453332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:38.457348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:38.482813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 199/393] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:38.487462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.561085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:38.599612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba4a932-a123-40e6-b783-b686b2656925 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:38.640948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94f871-d4aa-49ff-8a87-81ccd2bcabf3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:38.648174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30f260-d438-4f66-8169-01dd03155b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:38.650097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30f260-d438-4f66-8169-01dd03155b33 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.650620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30f260-d438-4f66-8169-01dd03155b33 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.651096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30f260-d438-4f66-8169-01dd03155b33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.673889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30f260-d438-4f66-8169-01dd03155b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-ad30f260-d438-4f66-8169-01dd03155b33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:38.683145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:38.684158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:38.684992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 195/394] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:38.686392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67c43-5a53-4347-bfa6-0b12bd33580f None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.686678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67c43-5a53-4347-bfa6-0b12bd33580f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.686950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a67c43-5a53-4347-bfa6-0b12bd33580f None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.692673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 196/395] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:38.692673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4df39-4c22-472e-b7a0-c0bd05f4f96d None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.692673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4df39-4c22-472e-b7a0-c0bd05f4f96d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.692673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4df39-4c22-472e-b7a0-c0bd05f4f96d None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.693410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e3610-2991-4ce3-bcac-c0827e2cfe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:38.694942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e3610-2991-4ce3-bcac-c0827e2cfe3d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.694942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e3610-2991-4ce3-bcac-c0827e2cfe3d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.694942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e3610-2991-4ce3-bcac-c0827e2cfe3d None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.695572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 197/396] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:38.696706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e3610-2991-4ce3-bcac-c0827e2cfe3d None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:38.697116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3843363-b925-4205-82a2-81528df837c4 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.697398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3843363-b925-4205-82a2-81528df837c4 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.697686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3843363-b925-4205-82a2-81528df837c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.698268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e3610-2991-4ce3-bcac-c0827e2cfe3d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:38.700613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 200/397] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:38.701212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e3610-2991-4ce3-bcac-c0827e2cfe3d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.718862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f3843363-b925-4205-82a2-81528df837c4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:38.720074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f3843363-b925-4205-82a2-81528df837c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:38.777374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:38.792858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 198/398] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:38.799230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ba85-7099-4006-b23d-b72d393d6fbc None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.799514 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ba85-7099-4006-b23d-b72d393d6fbc None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.799797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61ba85-7099-4006-b23d-b72d393d6fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.822969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ba61ba85-7099-4006-b23d-b72d393d6fbc None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:38.824221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ba61ba85-7099-4006-b23d-b72d393d6fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:38.826466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e3610-2991-4ce3-bcac-c0827e2cfe3d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:38.843483 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:38.879046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:38.880556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.881490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.882282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.883975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:38.884888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:38.888066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:38.903560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 201/399] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:38.905114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.914020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 199/400] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:38.915005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61ba85-7099-4006-b23d-b72d393d6fbc None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.079145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f385afe-80a1-4a2f-989a-dfdcf69ab5f1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.130896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61ba85-7099-4006-b23d-b72d393d6fbc None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.133259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05af755-c8ea-4f79-b988-e8a064f0a7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.133968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05af755-c8ea-4f79-b988-e8a064f0a7cd None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.134251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05af755-c8ea-4f79-b988-e8a064f0a7cd None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.134526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05af755-c8ea-4f79-b988-e8a064f0a7cd None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.136223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05af755-c8ea-4f79-b988-e8a064f0a7cd None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:39.137255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05af755-c8ea-4f79-b988-e8a064f0a7cd None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:39.139480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 202/401] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:39.140406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05af755-c8ea-4f79-b988-e8a064f0a7cd None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.154237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:39.175523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f6475-fbe7-4e4d-9a71-03b75d7144a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.176336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6475-fbe7-4e4d-9a71-03b75d7144a1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.176712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6475-fbe7-4e4d-9a71-03b75d7144a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.177077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f6475-fbe7-4e4d-9a71-03b75d7144a1 None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.178352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f6475-fbe7-4e4d-9a71-03b75d7144a1 None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:39.179903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f6475-fbe7-4e4d-9a71-03b75d7144a1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:39.186781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 200/402] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:39.187788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f6475-fbe7-4e4d-9a71-03b75d7144a1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.194035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:39.279623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05af755-c8ea-4f79-b988-e8a064f0a7cd None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.303966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f6475-fbe7-4e4d-9a71-03b75d7144a1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.322749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373f5f4-62a8-4ac8-a8f1-eccb54c3fb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.324747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373f5f4-62a8-4ac8-a8f1-eccb54c3fb11 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.325221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373f5f4-62a8-4ac8-a8f1-eccb54c3fb11 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.325703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373f5f4-62a8-4ac8-a8f1-eccb54c3fb11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.338423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373f5f4-62a8-4ac8-a8f1-eccb54c3fb11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-0373f5f4-62a8-4ac8-a8f1-eccb54c3fb11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:39.344081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:39.345827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 203/403] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:39.347363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373f5f4-62a8-4ac8-a8f1-eccb54c3fb11 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.359024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1ad2a-39c5-487c-b453-56f913d39618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.359695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1ad2a-39c5-487c-b453-56f913d39618 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.359977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1ad2a-39c5-487c-b453-56f913d39618 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.360247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1ad2a-39c5-487c-b453-56f913d39618 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/groups/dc0d5de3eb9945d788fd23bad20d05a9/roles/90097e8ce1394daba58b200de808ad59` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.362530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1ad2a-39c5-487c-b453-56f913d39618 None admin] RBAC: Authorizing `identity:create_grant(project_id=485f0f7bdc174746a34e4c85b269d3dd, group_id=dc0d5de3eb9945d788fd23bad20d05a9, role_id=90097e8ce1394daba58b200de808ad59)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:39.364036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1ad2a-39c5-487c-b453-56f913d39618 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:39.378510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 201/404] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 13 19:35:39 2022] PUT /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/groups/dc0d5de3eb9945d788fd23bad20d05a9/roles/90097e8ce1394daba58b200de808ad59 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:39.390974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1ad2a-39c5-487c-b453-56f913d39618 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.475071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:39.477647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:39.478859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373f5f4-62a8-4ac8-a8f1-eccb54c3fb11 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.524599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed13da-b172-4715-95f3-f10f82e35ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.524818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed13da-b172-4715-95f3-f10f82e35ec0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.525052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed13da-b172-4715-95f3-f10f82e35ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.525244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed13da-b172-4715-95f3-f10f82e35ec0 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.526773 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed13da-b172-4715-95f3-f10f82e35ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:39.528624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed13da-b172-4715-95f3-f10f82e35ec0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:39.530831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 204/405] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:39 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:39.536795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed13da-b172-4715-95f3-f10f82e35ec0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.561596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1ad2a-39c5-487c-b453-56f913d39618 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.602329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41393921-320b-4346-a87f-125107d25292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.603114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41393921-320b-4346-a87f-125107d25292 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.603495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41393921-320b-4346-a87f-125107d25292 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.603864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41393921-320b-4346-a87f-125107d25292 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.619533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41393921-320b-4346-a87f-125107d25292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-41393921-320b-4346-a87f-125107d25292 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:39.626302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:39.627354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 202/406] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:39.628364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41393921-320b-4346-a87f-125107d25292 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.691755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed13da-b172-4715-95f3-f10f82e35ec0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.729040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.729734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.730020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.730295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.730965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:39.731268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:39.732792 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:39.745805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 205/407] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:39.746858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.755035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41393921-320b-4346-a87f-125107d25292 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.770564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:39.786044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:39.814120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041f06f-136c-4a73-8077-f0acd851fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.815157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041f06f-136c-4a73-8077-f0acd851fc29 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.815679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041f06f-136c-4a73-8077-f0acd851fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.816130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041f06f-136c-4a73-8077-f0acd851fc29 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/groups/dc0d5de3eb9945d788fd23bad20d05a9/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.818676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041f06f-136c-4a73-8077-f0acd851fc29 None admin] RBAC: Authorizing `identity:create_grant(project_id=485f0f7bdc174746a34e4c85b269d3dd, group_id=dc0d5de3eb9945d788fd23bad20d05a9, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:39.821023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041f06f-136c-4a73-8077-f0acd851fc29 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:39.845867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 203/408] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 13 19:35:39 2022] PUT /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd/groups/dc0d5de3eb9945d788fd23bad20d05a9/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:39.847296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041f06f-136c-4a73-8077-f0acd851fc29 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:39.883876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726120c6-39c0-4658-ab55-1ce85d9ed8b6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:39.947666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:39.948326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:39.948647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:39.948928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:39.949698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:39.950011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:39.951442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:39.973789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 206/409] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:39.984478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:40.080220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041f06f-136c-4a73-8077-f0acd851fc29 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:40.146690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd247292-a2ec-41dd-bb3f-836fbdd2f91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:40.148040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd247292-a2ec-41dd-bb3f-836fbdd2f91e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:40.148605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd247292-a2ec-41dd-bb3f-836fbdd2f91e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:40.149027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd247292-a2ec-41dd-bb3f-836fbdd2f91e None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:40.150606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd247292-a2ec-41dd-bb3f-836fbdd2f91e None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:40.152301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd247292-a2ec-41dd-bb3f-836fbdd2f91e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:40.155635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 204/410] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:39 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:40.161096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd247292-a2ec-41dd-bb3f-836fbdd2f91e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:40.170876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:40.182212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262791bb-3498-43b3-8bbf-c70aa46adfdd None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:40.206645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:40.229330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a006b1a-6e47-45be-bcdc-e37db556419c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:40.231202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006b1a-6e47-45be-bcdc-e37db556419c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:40.231757 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006b1a-6e47-45be-bcdc-e37db556419c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:40.232177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a006b1a-6e47-45be-bcdc-e37db556419c None admin] PATH_INFO: `/v3/groups/0df7c14ac803403399c7d9ace881a4dc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:40.233657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a006b1a-6e47-45be-bcdc-e37db556419c None admin] RBAC: Authorizing `identity:get_group(group_id=0df7c14ac803403399c7d9ace881a4dc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:40.235295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a006b1a-6e47-45be-bcdc-e37db556419c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:40.240737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 207/411] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:39 2022] GET /identity/v3/groups/0df7c14ac803403399c7d9ace881a4dc => generated 229 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:40.243925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a006b1a-6e47-45be-bcdc-e37db556419c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:40.290977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd247292-a2ec-41dd-bb3f-836fbdd2f91e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:40.338293 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91e11f-4f7d-4475-9c53-40fc19c33757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:40.338854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e11f-4f7d-4475-9c53-40fc19c33757 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:40.339829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e11f-4f7d-4475-9c53-40fc19c33757 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:40.340078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e11f-4f7d-4475-9c53-40fc19c33757 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:40.341299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91e11f-4f7d-4475-9c53-40fc19c33757 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:40.343542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91e11f-4f7d-4475-9c53-40fc19c33757 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:40.349640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 205/412] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:40 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:40.351791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91e11f-4f7d-4475-9c53-40fc19c33757 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:40.375540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a006b1a-6e47-45be-bcdc-e37db556419c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:40.423317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01657dbe-b2f0-48bf-9dc8-8702421ad307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:40.423965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01657dbe-b2f0-48bf-9dc8-8702421ad307 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:40.424245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01657dbe-b2f0-48bf-9dc8-8702421ad307 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:40.424517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01657dbe-b2f0-48bf-9dc8-8702421ad307 None admin] PATH_INFO: `/v3/projects/61d39e602be54c45a9fb48df499e3bc7` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:40.425870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01657dbe-b2f0-48bf-9dc8-8702421ad307 None admin] RBAC: Authorizing `identity:get_project(project_id=61d39e602be54c45a9fb48df499e3bc7)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:40.428114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01657dbe-b2f0-48bf-9dc8-8702421ad307 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:40.433651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 208/413] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:40 2022] GET /identity/v3/projects/61d39e602be54c45a9fb48df499e3bc7 => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:40.442977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01657dbe-b2f0-48bf-9dc8-8702421ad307 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:40.507548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:40.524785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91e11f-4f7d-4475-9c53-40fc19c33757 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:40.531470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:40.620103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd1759-2745-45bc-bbdc-cb1e390a13d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:40.621315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1759-2745-45bc-bbdc-cb1e390a13d4 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:40.621718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1759-2745-45bc-bbdc-cb1e390a13d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:40.622166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd1759-2745-45bc-bbdc-cb1e390a13d4 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/groups/dc0d5de3eb9945d788fd23bad20d05a9/roles/90097e8ce1394daba58b200de808ad59` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:40.630310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd1759-2745-45bc-bbdc-cb1e390a13d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9b796cfb79246dca28d73123bc5e6bc, group_id=dc0d5de3eb9945d788fd23bad20d05a9, role_id=90097e8ce1394daba58b200de808ad59)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:40.633014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd1759-2745-45bc-bbdc-cb1e390a13d4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:40.665765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 206/414] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 13 19:35:40 2022] PUT /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc/groups/dc0d5de3eb9945d788fd23bad20d05a9/roles/90097e8ce1394daba58b200de808ad59 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:40.680573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01657dbe-b2f0-48bf-9dc8-8702421ad307 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:40.719064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f5e02-a427-4572-b253-7b578c19c36d None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:40.719351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f5e02-a427-4572-b253-7b578c19c36d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:40.719628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f5e02-a427-4572-b253-7b578c19c36d None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:40.721374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 207/415] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:40.731686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3992319-b9ac-4742-8654-1255b82b77d3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:40.732021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3992319-b9ac-4742-8654-1255b82b77d3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:40.732235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3992319-b9ac-4742-8654-1255b82b77d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:40.748969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461557c-8544-48d1-8d35-5977c3287f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:40.749688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461557c-8544-48d1-8d35-5977c3287f40 None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:40.749976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461557c-8544-48d1-8d35-5977c3287f40 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:40.750558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461557c-8544-48d1-8d35-5977c3287f40 None admin] PATH_INFO: `/v3/projects/61d39e602be54c45a9fb48df499e3bc7/groups/0df7c14ac803403399c7d9ace881a4dc/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:40.753788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461557c-8544-48d1-8d35-5977c3287f40 None admin] RBAC: Authorizing `identity:create_grant(project_id=61d39e602be54c45a9fb48df499e3bc7, group_id=0df7c14ac803403399c7d9ace881a4dc, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:40.760724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461557c-8544-48d1-8d35-5977c3287f40 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:40.771512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f3992319-b9ac-4742-8654-1255b82b77d3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:40.772898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f3992319-b9ac-4742-8654-1255b82b77d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:40.785363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 209/416] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Sep 13 19:35:40 2022] PUT /identity/v3/projects/61d39e602be54c45a9fb48df499e3bc7/groups/0df7c14ac803403399c7d9ace881a4dc/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:40.899511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 208/417] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:41.318102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461557c-8544-48d1-8d35-5977c3287f40 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:41.438224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:41.463662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461557c-8544-48d1-8d35-5977c3287f40 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:41.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06300e06-36fc-4fb2-9643-c15a91df5ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:41.509946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06300e06-36fc-4fb2-9643-c15a91df5ddc None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.510225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06300e06-36fc-4fb2-9643-c15a91df5ddc None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.510494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06300e06-36fc-4fb2-9643-c15a91df5ddc None admin] PATH_INFO: `/v3/roles/member` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.524786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06300e06-36fc-4fb2-9643-c15a91df5ddc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-06300e06-36fc-4fb2-9643-c15a91df5ddc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:41.530435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:41.531365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:41.532214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 210/418] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:41.539743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3992319-b9ac-4742-8654-1255b82b77d3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:41.647310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:41.659515 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3992319-b9ac-4742-8654-1255b82b77d3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:41.695764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c7dad-4474-40bf-bc72-943772db5e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:41.696642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c7dad-4474-40bf-bc72-943772db5e61 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.696937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c7dad-4474-40bf-bc72-943772db5e61 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.697208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c7dad-4474-40bf-bc72-943772db5e61 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.697929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c7dad-4474-40bf-bc72-943772db5e61 None admin] RBAC: Adding query filter params (name=member) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:41.698898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c7dad-4474-40bf-bc72-943772db5e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:41.700438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c7dad-4474-40bf-bc72-943772db5e61 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:41.710204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 209/419] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:41.718195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06300e06-36fc-4fb2-9643-c15a91df5ddc None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:41.857080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06300e06-36fc-4fb2-9643-c15a91df5ddc None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:41.907295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1ef4a-e95f-4ebf-a9cc-747013ced972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:41.907858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1ef4a-e95f-4ebf-a9cc-747013ced972 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.908144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1ef4a-e95f-4ebf-a9cc-747013ced972 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.908422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1ef4a-e95f-4ebf-a9cc-747013ced972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.934369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:42.003942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1ef4a-e95f-4ebf-a9cc-747013ced972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.005546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b1ef4a-e95f-4ebf-a9cc-747013ced972 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.048670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba63f2-7976-437e-9c99-8c3a79ce3842 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.049016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba63f2-7976-437e-9c99-8c3a79ce3842 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.049224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba63f2-7976-437e-9c99-8c3a79ce3842 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.050600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 210/420] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:42.057205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3df8e-81c2-42a6-a8a4-657609ff10fa None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.058531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3df8e-81c2-42a6-a8a4-657609ff10fa None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.058932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3df8e-81c2-42a6-a8a4-657609ff10fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.080736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-42b3df8e-81c2-42a6-a8a4-657609ff10fa None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:42.083047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-42b3df8e-81c2-42a6-a8a4-657609ff10fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:42.188069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 211/421] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:35:42.196297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1ef4a-e95f-4ebf-a9cc-747013ced972 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.228621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 211/422] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:42.261011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:42.338573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1ef4a-e95f-4ebf-a9cc-747013ced972 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.398315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad821-96b5-4f3f-8f2a-28f3a83717a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.399755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad821-96b5-4f3f-8f2a-28f3a83717a5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.400392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad821-96b5-4f3f-8f2a-28f3a83717a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.401073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad821-96b5-4f3f-8f2a-28f3a83717a5 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.403481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ad821-96b5-4f3f-8f2a-28f3a83717a5 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.406337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ad821-96b5-4f3f-8f2a-28f3a83717a5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.409956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 212/423] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:42.417466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3df8e-81c2-42a6-a8a4-657609ff10fa None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.546595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:42.570461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3df8e-81c2-42a6-a8a4-657609ff10fa None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.578494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad821-96b5-4f3f-8f2a-28f3a83717a5 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.609658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22269b4-e4e1-460b-9419-dcc4033d3af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.610302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22269b4-e4e1-460b-9419-dcc4033d3af1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.610583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22269b4-e4e1-460b-9419-dcc4033d3af1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.610853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22269b4-e4e1-460b-9419-dcc4033d3af1 None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.612025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22269b4-e4e1-460b-9419-dcc4033d3af1 None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.616607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22269b4-e4e1-460b-9419-dcc4033d3af1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.616607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 212/424] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:42.621084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22269b4-e4e1-460b-9419-dcc4033d3af1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.731786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22269b4-e4e1-460b-9419-dcc4033d3af1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.768327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.769215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.769658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.770069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.770931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] RBAC: Adding query filter params (group.id=dc0d5de3eb9945d788fd23bad20d05a9, role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:42.771393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.772961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.779950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 213/425] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/role_assignments?group.id=dc0d5de3eb9945d788fd23bad20d05a9&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:42.804203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad821-96b5-4f3f-8f2a-28f3a83717a5 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.850010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:42.890966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7da800-9b74-49c7-b8f3-d56518980569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.892045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7da800-9b74-49c7-b8f3-d56518980569 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.892555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7da800-9b74-49c7-b8f3-d56518980569 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.893047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7da800-9b74-49c7-b8f3-d56518980569 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.923392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7da800-9b74-49c7-b8f3-d56518980569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-de7da800-9b74-49c7-b8f3-d56518980569 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:42.933797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:42.934832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:42.935669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:42.936282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 213/426] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:42.941797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.975967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:43.033258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882da3f-31d0-4ddd-8b42-3a19a7bd95cd None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.033857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882da3f-31d0-4ddd-8b42-3a19a7bd95cd None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.034338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882da3f-31d0-4ddd-8b42-3a19a7bd95cd None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.036863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 214/427] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:43.042168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8923a-63cf-4a99-8720-74f5a7b5716b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.042593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8923a-63cf-4a99-8720-74f5a7b5716b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.042721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8923a-63cf-4a99-8720-74f5a7b5716b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.079699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8cb8923a-63cf-4a99-8720-74f5a7b5716b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:43.081309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8cb8923a-63cf-4a99-8720-74f5a7b5716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:43.101596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0580a-c8e2-4322-9b26-93025d400fa6 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:43.153095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:43.153749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.154642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.154935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.155749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:43.156211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:43.157910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:43.171801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 214/428] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:43.172981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 215/429] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:43.177129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:43.281395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:43.306262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ebcf5-1ff5-42a3-8647-fea9a7bdd306 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:43.343496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ae81b-52dd-4e1c-bf2c-9be53ba41213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:43.344435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ae81b-52dd-4e1c-bf2c-9be53ba41213 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.344914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ae81b-52dd-4e1c-bf2c-9be53ba41213 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.345414 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1ae81b-52dd-4e1c-bf2c-9be53ba41213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.418800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ae81b-52dd-4e1c-bf2c-9be53ba41213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:43.420618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1ae81b-52dd-4e1c-bf2c-9be53ba41213 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:43.503956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:43.530101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 215/430] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:43.538851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8923a-63cf-4a99-8720-74f5a7b5716b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:43.622235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ae81b-52dd-4e1c-bf2c-9be53ba41213 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:43.653713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8923a-63cf-4a99-8720-74f5a7b5716b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:43.662289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:43.694438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754dac1-b820-4649-ab41-4034f987cb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:43.695583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754dac1-b820-4649-ab41-4034f987cb18 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.696142 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754dac1-b820-4649-ab41-4034f987cb18 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.696891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754dac1-b820-4649-ab41-4034f987cb18 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.699439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754dac1-b820-4649-ab41-4034f987cb18 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:43.702217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754dac1-b820-4649-ab41-4034f987cb18 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:43.704270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 216/431] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:43 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:43.705530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395fe39-db62-4de2-8722-8304f84c085b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.705846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395fe39-db62-4de2-8722-8304f84c085b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.706122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395fe39-db62-4de2-8722-8304f84c085b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.707469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 217/432] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:43.710482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395fe39-db62-4de2-8722-8304f84c085b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:43.743946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1ae81b-52dd-4e1c-bf2c-9be53ba41213 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:43.791535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8a163-f766-4256-a4a8-341db9a5f3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:43.792202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a163-f766-4256-a4a8-341db9a5f3e2 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.792481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a163-f766-4256-a4a8-341db9a5f3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.792753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8a163-f766-4256-a4a8-341db9a5f3e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.803286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8a163-f766-4256-a4a8-341db9a5f3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-bde8a163-f766-4256-a4a8-341db9a5f3e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:43.808776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:43.809875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:43.810975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 216/433] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:43.811902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb190c-9ede-405e-b07c-40397c922f96 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.811902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb190c-9ede-405e-b07c-40397c922f96 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.811902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb190c-9ede-405e-b07c-40397c922f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.830594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-54cb190c-9ede-405e-b07c-40397c922f96 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:43.831818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-54cb190c-9ede-405e-b07c-40397c922f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:43.847627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395fe39-db62-4de2-8722-8304f84c085b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:43.872006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:43.888124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bd193-9cc9-434f-8009-ab580b4f2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:43.888688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bd193-9cc9-434f-8009-ab580b4f2177 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:43.888968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bd193-9cc9-434f-8009-ab580b4f2177 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:43.889243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bd193-9cc9-434f-8009-ab580b4f2177 None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:43.890383 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bd193-9cc9-434f-8009-ab580b4f2177 None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:43.891815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bd193-9cc9-434f-8009-ab580b4f2177 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:43.893567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 218/434] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:43 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:43.894181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bd193-9cc9-434f-8009-ab580b4f2177 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:43.918043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 217/435] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:43.918715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb190c-9ede-405e-b07c-40397c922f96 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:43.947201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:44.032842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bd193-9cc9-434f-8009-ab580b4f2177 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.038021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb190c-9ede-405e-b07c-40397c922f96 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.077085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.077727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.078012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.078287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.078961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] RBAC: Adding query filter params (name=member) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:44.079322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:44.079974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ce111-f3b8-4ca4-9132-bb5d18f00065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.080572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ce111-f3b8-4ca4-9132-bb5d18f00065 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.080681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:44.080853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ce111-f3b8-4ca4-9132-bb5d18f00065 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.081128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ce111-f3b8-4ca4-9132-bb5d18f00065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.081886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ce111-f3b8-4ca4-9132-bb5d18f00065 None admin] RBAC: Adding query filter params (group.id=dc0d5de3eb9945d788fd23bad20d05a9, role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:44.082212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ce111-f3b8-4ca4-9132-bb5d18f00065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:44.083668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ce111-f3b8-4ca4-9132-bb5d18f00065 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:44.097578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 218/436] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:43 2022] GET /identity/v3/role_assignments?group.id=dc0d5de3eb9945d788fd23bad20d05a9&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=90097e8ce1394daba58b200de808ad59 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:44.097578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd3bf-3014-43e3-97a6-b1f8537b4d0c None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.097578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd3bf-3014-43e3-97a6-b1f8537b4d0c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.097578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11edd3bf-3014-43e3-97a6-b1f8537b4d0c None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.098007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 219/437] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:44.098007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:44.098131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 219/438] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:44.099544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082c25-2574-4069-9e5b-df06035356db None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.099975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082c25-2574-4069-9e5b-df06035356db None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.100124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a082c25-2574-4069-9e5b-df06035356db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.123008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-1a082c25-2574-4069-9e5b-df06035356db None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:44.124259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1a082c25-2574-4069-9e5b-df06035356db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:44.162996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:44.219951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 220/439] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:44.220920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a082c25-2574-4069-9e5b-df06035356db None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:44.262399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed26d90e-4400-4351-9b89-ee894abdf287 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.266472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:44.311304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4120b7-fdc0-4b83-b1b6-d41b7d791868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.312088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4120b7-fdc0-4b83-b1b6-d41b7d791868 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.312587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4120b7-fdc0-4b83-b1b6-d41b7d791868 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.312873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4120b7-fdc0-4b83-b1b6-d41b7d791868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.378731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a082c25-2574-4069-9e5b-df06035356db None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.413459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4120b7-fdc0-4b83-b1b6-d41b7d791868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:44.414950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4120b7-fdc0-4b83-b1b6-d41b7d791868 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:44.426901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18098af2-6df2-4b1b-b3a2-c73973305de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.427533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098af2-6df2-4b1b-b3a2-c73973305de3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.427811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098af2-6df2-4b1b-b3a2-c73973305de3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.428081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18098af2-6df2-4b1b-b3a2-c73973305de3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.438887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18098af2-6df2-4b1b-b3a2-c73973305de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-18098af2-6df2-4b1b-b3a2-c73973305de3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:44.443703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:44.444797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:44.445858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 221/440] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:44.452250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18098af2-6df2-4b1b-b3a2-c73973305de3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:44.481310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:44.536006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 220/441] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:44.536851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4120b7-fdc0-4b83-b1b6-d41b7d791868 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:44.542133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:44.561164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18098af2-6df2-4b1b-b3a2-c73973305de3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.596019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.596625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.596904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.597182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.597885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:44.598203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:44.599558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:44.608025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 222/442] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:44.608025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:44.641918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4120b7-fdc0-4b83-b1b6-d41b7d791868 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.675334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a4bbf-cb76-4633-b2ef-1449d0c59bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.675887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4bbf-cb76-4633-b2ef-1449d0c59bbd None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.676173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4bbf-cb76-4633-b2ef-1449d0c59bbd None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.676450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a4bbf-cb76-4633-b2ef-1449d0c59bbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.687696 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a4bbf-cb76-4633-b2ef-1449d0c59bbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:44.691707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-154a4bbf-cb76-4633-b2ef-1449d0c59bbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:44.692666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:44.694150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 221/443] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:44.695389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a4bbf-cb76-4633-b2ef-1449d0c59bbd None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:44.708694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a358c8-18eb-48c9-b801-86bf48962ab8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.744051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03f38-d0d3-4534-bcc5-7f8e87236367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.744686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03f38-d0d3-4534-bcc5-7f8e87236367 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.744968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03f38-d0d3-4534-bcc5-7f8e87236367 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.745283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03f38-d0d3-4534-bcc5-7f8e87236367 None admin] PATH_INFO: `/v3/projects/485f0f7bdc174746a34e4c85b269d3dd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.746454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03f38-d0d3-4534-bcc5-7f8e87236367 None admin] RBAC: Authorizing `identity:get_project(project_id=485f0f7bdc174746a34e4c85b269d3dd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:44.747898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03f38-d0d3-4534-bcc5-7f8e87236367 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:44.749527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 223/444] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:44 2022] GET /identity/v3/projects/485f0f7bdc174746a34e4c85b269d3dd => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:44.749527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03f38-d0d3-4534-bcc5-7f8e87236367 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:44.759847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:44.798012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a4bbf-cb76-4633-b2ef-1449d0c59bbd None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.832214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba783f26-f1bf-4ce8-bc7d-a9191031c249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.832821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba783f26-f1bf-4ce8-bc7d-a9191031c249 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.833107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba783f26-f1bf-4ce8-bc7d-a9191031c249 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.833382 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba783f26-f1bf-4ce8-bc7d-a9191031c249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.851766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03f38-d0d3-4534-bcc5-7f8e87236367 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:44.889629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:44.890244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.890521 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.890802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.891461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:44.891765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:44.893112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:44.901088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 224/445] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:44.901997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba783f26-f1bf-4ce8-bc7d-a9191031c249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:44.902435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:44.903470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba783f26-f1bf-4ce8-bc7d-a9191031c249 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:44.917826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:44.967398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.002631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa7af-adeb-4b23-87d5-9752515b985a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.010872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 222/446] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:45.010872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba783f26-f1bf-4ce8-bc7d-a9191031c249 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.039768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911577a3-a7b5-4938-bf3c-d7f32b599490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.040398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911577a3-a7b5-4938-bf3c-d7f32b599490 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.040677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911577a3-a7b5-4938-bf3c-d7f32b599490 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.040950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911577a3-a7b5-4938-bf3c-d7f32b599490 None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.042159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911577a3-a7b5-4938-bf3c-d7f32b599490 None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.043602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911577a3-a7b5-4938-bf3c-d7f32b599490 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.045181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 225/447] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:44 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:45.045964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911577a3-a7b5-4938-bf3c-d7f32b599490 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.112870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba783f26-f1bf-4ce8-bc7d-a9191031c249 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.128799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.147783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911577a3-a7b5-4938-bf3c-d7f32b599490 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.148721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b111d83-1c33-4ae5-be93-095257d663ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.149326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b111d83-1c33-4ae5-be93-095257d663ee None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.149601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b111d83-1c33-4ae5-be93-095257d663ee None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.150605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b111d83-1c33-4ae5-be93-095257d663ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.179564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.181496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a5963-820f-4819-b310-f0c6a882f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.182189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a5963-820f-4819-b310-f0c6a882f073 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.183039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a5963-820f-4819-b310-f0c6a882f073 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.183307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a5963-820f-4819-b310-f0c6a882f073 None admin] PATH_INFO: `/v3/projects/c9b796cfb79246dca28d73123bc5e6bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.185276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a5963-820f-4819-b310-f0c6a882f073 None admin] RBAC: Authorizing `identity:get_project(project_id=c9b796cfb79246dca28d73123bc5e6bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.186776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a5963-820f-4819-b310-f0c6a882f073 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.188439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 226/448] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/projects/c9b796cfb79246dca28d73123bc5e6bc => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:45.189637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a5963-820f-4819-b310-f0c6a882f073 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.219442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b111d83-1c33-4ae5-be93-095257d663ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.220849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b111d83-1c33-4ae5-be93-095257d663ee None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.293437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a5963-820f-4819-b310-f0c6a882f073 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.323363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 223/449] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:45.324193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b111d83-1c33-4ae5-be93-095257d663ee None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.330541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.331154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.331434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.331707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.332441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] RBAC: Adding query filter params (group.id=dc0d5de3eb9945d788fd23bad20d05a9, role.id=afc24e28d6c34708ab897fc4b7029bb2, scope.project.id=485f0f7bdc174746a34e4c85b269d3dd) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:45.332745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.334231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.340409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 227/450] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/role_assignments?group.id=dc0d5de3eb9945d788fd23bad20d05a9&scope.project.id=485f0f7bdc174746a34e4c85b269d3dd&role.id=afc24e28d6c34708ab897fc4b7029bb2 => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:45.341111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.364015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.397828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.442296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b111d83-1c33-4ae5-be93-095257d663ee None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.474412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175f6b4-b98c-424c-b6b6-26f273b19fa0 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.491540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6a4c9-8c0c-4dc2-a9ef-4d09078df7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.492458 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a4c9-8c0c-4dc2-a9ef-4d09078df7ee None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.492864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a4c9-8c0c-4dc2-a9ef-4d09078df7ee None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.493258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6a4c9-8c0c-4dc2-a9ef-4d09078df7ee None admin] PATH_INFO: `/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.494806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6a4c9-8c0c-4dc2-a9ef-4d09078df7ee None admin] RBAC: Authorizing `identity:get_group(group_id=dc0d5de3eb9945d788fd23bad20d05a9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.497426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6a4c9-8c0c-4dc2-a9ef-4d09078df7ee None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.504536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 224/451] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/groups/dc0d5de3eb9945d788fd23bad20d05a9 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:45.505484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6a4c9-8c0c-4dc2-a9ef-4d09078df7ee None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.524800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f5e16-06fa-4e8c-ae02-94965476c81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.525095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f5e16-06fa-4e8c-ae02-94965476c81d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.525246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f5e16-06fa-4e8c-ae02-94965476c81d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.525451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f5e16-06fa-4e8c-ae02-94965476c81d None admin] PATH_INFO: `/v3/projects/61d39e602be54c45a9fb48df499e3bc7` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.526781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f5e16-06fa-4e8c-ae02-94965476c81d None admin] RBAC: Authorizing `identity:get_project(project_id=61d39e602be54c45a9fb48df499e3bc7)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.528143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f5e16-06fa-4e8c-ae02-94965476c81d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.534319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 228/452] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/projects/61d39e602be54c45a9fb48df499e3bc7 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:45.536087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f5e16-06fa-4e8c-ae02-94965476c81d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.638882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6a4c9-8c0c-4dc2-a9ef-4d09078df7ee None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.643070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.666196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f5e16-06fa-4e8c-ae02-94965476c81d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.674921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.684123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.684123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.684123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.684123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.684489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] RBAC: Adding query filter params (group.id=dc0d5de3eb9945d788fd23bad20d05a9, role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=c9b796cfb79246dca28d73123bc5e6bc) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:45.684489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.687501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.691976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 225/453] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/role_assignments?group.id=dc0d5de3eb9945d788fd23bad20d05a9&scope.project.id=c9b796cfb79246dca28d73123bc5e6bc&role.id=90097e8ce1394daba58b200de808ad59 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:45.715211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a04d9b-18bf-44ce-9801-49ec1411f1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.715906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a04d9b-18bf-44ce-9801-49ec1411f1c9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.716152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a04d9b-18bf-44ce-9801-49ec1411f1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.716427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a04d9b-18bf-44ce-9801-49ec1411f1c9 None admin] PATH_INFO: `/v3/groups/0df7c14ac803403399c7d9ace881a4dc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.717780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a04d9b-18bf-44ce-9801-49ec1411f1c9 None admin] RBAC: Authorizing `identity:get_group(group_id=0df7c14ac803403399c7d9ace881a4dc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.719272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a04d9b-18bf-44ce-9801-49ec1411f1c9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.721155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 229/454] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/groups/0df7c14ac803403399c7d9ace881a4dc => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:45.727347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.861247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20407eb-b2ec-4d8f-8c5a-493d95ad1439 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.896158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb602a-fb6b-40f2-9eb7-e61b837f09a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.896761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb602a-fb6b-40f2-9eb7-e61b837f09a4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.897038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb602a-fb6b-40f2-9eb7-e61b837f09a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.897306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb602a-fb6b-40f2-9eb7-e61b837f09a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.898058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb602a-fb6b-40f2-9eb7-e61b837f09a4 None admin] RBAC: Adding query filter params (group.id=0df7c14ac803403399c7d9ace881a4dc, role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=61d39e602be54c45a9fb48df499e3bc7) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:45.898363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb602a-fb6b-40f2-9eb7-e61b837f09a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.899805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb602a-fb6b-40f2-9eb7-e61b837f09a4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.906632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 226/455] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/role_assignments?group.id=0df7c14ac803403399c7d9ace881a4dc&scope.project.id=61d39e602be54c45a9fb48df499e3bc7&role.id=b698800ceec3460a8519ea627650da0f => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:48.720370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74d61c-57da-4c19-9fc0-79a8d357b68a None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:48.721394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74d61c-57da-4c19-9fc0-79a8d357b68a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:48.721916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74d61c-57da-4c19-9fc0-79a8d357b68a None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:48.723723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 230/456] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:48.730423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e62286-fe81-4564-b1f6-41e785a5cf27 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:48.731271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e62286-fe81-4564-b1f6-41e785a5cf27 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:48.731740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e62286-fe81-4564-b1f6-41e785a5cf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:48.768602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-26e62286-fe81-4564-b1f6-41e785a5cf27 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:48.770286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-26e62286-fe81-4564-b1f6-41e785a5cf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:48.876502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:48.900806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 227/457] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:49.055693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb7c2-9fbf-466c-8a0c-35d73fe4a448 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.056631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb7c2-9fbf-466c-8a0c-35d73fe4a448 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.057903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfb7c2-9fbf-466c-8a0c-35d73fe4a448 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.060641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 231/458] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:49.069478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464898e-0835-493b-b4d5-cfbda2f225ab None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.070408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464898e-0835-493b-b4d5-cfbda2f225ab None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.071420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464898e-0835-493b-b4d5-cfbda2f225ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.109923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f41f-21f3-46f2-bccf-c50398d282cf None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.110321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f41f-21f3-46f2-bccf-c50398d282cf None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.110476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1f41f-21f3-46f2-bccf-c50398d282cf None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.112024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 232/459] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:49.112616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3464898e-0835-493b-b4d5-cfbda2f225ab None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:49.115028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3464898e-0835-493b-b4d5-cfbda2f225ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:49.117083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f59a-a90f-4a69-a3a3-b34bba36b64a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.117370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f59a-a90f-4a69-a3a3-b34bba36b64a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.117674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f59a-a90f-4a69-a3a3-b34bba36b64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.140927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bcc9f59a-a90f-4a69-a3a3-b34bba36b64a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:49.142161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bcc9f59a-a90f-4a69-a3a3-b34bba36b64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:49.241216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 228/460] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:49.243634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 233/461] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:49.523970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464898e-0835-493b-b4d5-cfbda2f225ab None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:49.609357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d0810-8d0a-4efa-a09e-cb64738947a1 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.609825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d0810-8d0a-4efa-a09e-cb64738947a1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.610221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d0810-8d0a-4efa-a09e-cb64738947a1 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.611878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 234/462] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:49.616978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4446f9b-2414-4262-b072-2bbbbc3a097e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.617361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4446f9b-2414-4262-b072-2bbbbc3a097e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.617762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4446f9b-2414-4262-b072-2bbbbc3a097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.635805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464898e-0835-493b-b4d5-cfbda2f225ab None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:49.639559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f4446f9b-2414-4262-b072-2bbbbc3a097e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:49.641176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f4446f9b-2414-4262-b072-2bbbbc3a097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:49.660984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:49.696993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:49.697659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.698004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.698281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.708789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:49.710430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:49.726884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:49.727828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:49.728712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:49.728712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:49.728712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:49.728712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 229/463] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:49.729199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:49.733139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 235/464] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:49.734518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4446f9b-2414-4262-b072-2bbbbc3a097e None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:49.749522 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:49.848676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a7cf6-0f68-4e6b-b129-8b8fb284a64e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:49.854660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4446f9b-2414-4262-b072-2bbbbc3a097e None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:49.893589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:49.894572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.894683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.894949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.908982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:49.911370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:49.917049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:49.917965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.918464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.918952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.920407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:49.921098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:49.924931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:49.933066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:49.934016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:49.934878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:49.934878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:49.934878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:49.941654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 230/465] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:49.941654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:49.951301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 236/466] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:49.952879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:49.973048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.040114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.067313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2a657-17e4-49c4-9dc4-4541ab38ffd7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.075936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21196f-e2c8-4222-ac20-f2a8ade9999e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.107077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.108201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.108653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.109085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.111419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.112038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.112304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.112575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.113218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:50.113523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.114924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.122955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.125519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.127210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 237/467] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:50.128000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:50.148445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:50.149415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:50.150298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:50.150298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:50.150298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:50.150298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 231/468] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:50.150528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.239843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e95fc8-1b86-4da3-b7a7-f4b55ea71116 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.240148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.264985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b840f-e5c8-4390-a07a-a2feaf6eee56 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.271265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.279202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e7b04-a94e-40b8-b7e6-d73370e13940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.279808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7b04-a94e-40b8-b7e6-d73370e13940 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.280303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7b04-a94e-40b8-b7e6-d73370e13940 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.280410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e7b04-a94e-40b8-b7e6-d73370e13940 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.280988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e7b04-a94e-40b8-b7e6-d73370e13940 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.282451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e7b04-a94e-40b8-b7e6-d73370e13940 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.302301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.302941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.303233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.303495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.306376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 238/469] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:50 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:50.307697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e7b04-a94e-40b8-b7e6-d73370e13940 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.318454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.320384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:50.343632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:50.344598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:50.345506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:50.345506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:50.345506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:50.347187 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 232/470] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:50.347871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.459567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e7b04-a94e-40b8-b7e6-d73370e13940 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.498248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161a2be-735e-437f-ac60-0ad07a3fa31c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.504556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64eba0-70c4-4621-b903-00951245d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.505168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64eba0-70c4-4621-b903-00951245d40e None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.505443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64eba0-70c4-4621-b903-00951245d40e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.505747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64eba0-70c4-4621-b903-00951245d40e None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.506501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64eba0-70c4-4621-b903-00951245d40e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.507980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64eba0-70c4-4621-b903-00951245d40e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.531694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.535427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 239/471] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:50 2022] POST /identity/v3/users => generated 248 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:50.535602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64eba0-70c4-4621-b903-00951245d40e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.543970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.545050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.545562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.546099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.547040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:50.547456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.549121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.570832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 233/472] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:50.577080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.596364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.677106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64eba0-70c4-4621-b903-00951245d40e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.728580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23838a89-bee3-457d-9c2f-5181d3e9004e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.740833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29450243-053b-49e4-8e5d-1a1d142fbace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.741645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450243-053b-49e4-8e5d-1a1d142fbace None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.742029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450243-053b-49e4-8e5d-1a1d142fbace None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.742313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29450243-053b-49e4-8e5d-1a1d142fbace None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.743036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29450243-053b-49e4-8e5d-1a1d142fbace None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:50.743347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29450243-053b-49e4-8e5d-1a1d142fbace None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.744880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29450243-053b-49e4-8e5d-1a1d142fbace None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.757594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 240/473] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:50.758833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428e25d-1956-41a9-99bb-248ea8a95dcb None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.759156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428e25d-1956-41a9-99bb-248ea8a95dcb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.759399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428e25d-1956-41a9-99bb-248ea8a95dcb None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.761438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 241/474] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:50.765732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428e25d-1956-41a9-99bb-248ea8a95dcb None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.792119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d8a4a9-7c64-49f8-9453-b89175c54cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.792839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a4a9-7c64-49f8-9453-b89175c54cdb None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.793129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a4a9-7c64-49f8-9453-b89175c54cdb None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.793402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d8a4a9-7c64-49f8-9453-b89175c54cdb None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.794093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8a4a9-7c64-49f8-9453-b89175c54cdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.795560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d8a4a9-7c64-49f8-9453-b89175c54cdb None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.834062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 234/475] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:50 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:50.834718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5033e8-da1a-4a1d-94e0-d7a01cb14c38 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.835010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5033e8-da1a-4a1d-94e0-d7a01cb14c38 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.835287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5033e8-da1a-4a1d-94e0-d7a01cb14c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.863552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-5d5033e8-da1a-4a1d-94e0-d7a01cb14c38 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:50.865695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5d5033e8-da1a-4a1d-94e0-d7a01cb14c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:50.925679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428e25d-1956-41a9-99bb-248ea8a95dcb None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.928307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.934083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.995927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 235/476] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:50.997372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f50e6d-cde8-4845-b7bf-1b647b59c635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.998020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f50e6d-cde8-4845-b7bf-1b647b59c635 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.998291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f50e6d-cde8-4845-b7bf-1b647b59c635 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.998616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f50e6d-cde8-4845-b7bf-1b647b59c635 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.999273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f50e6d-cde8-4845-b7bf-1b647b59c635 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:51.000823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f50e6d-cde8-4845-b7bf-1b647b59c635 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:51.036842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 242/477] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:50 2022] POST /identity/v3/users => generated 249 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:51.355504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5033e8-da1a-4a1d-94e0-d7a01cb14c38 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:51.539012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5033e8-da1a-4a1d-94e0-d7a01cb14c38 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:51.604441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:51.605494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:51.606089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:51.606529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:51.607789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:51.610445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:51.622882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.sql.core [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:35:51.622882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 19:35:51.622882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [parameters: {'id': '1b35dff5dd8f46cb9f47ca5becca9e77', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 19:35:51.622882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88442) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application result.read() Sep 13 19:35:51.650817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:35:51.651942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application yield resource Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise self.value Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application session.commit() Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:35:51.652837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:35:51.653744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:35:51.654633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application result.read() Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:35:51.657091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [parameters: {'id': '1b35dff5dd8f46cb9f47ca5becca9e77', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:51.662895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 19:35:51.665005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:51.667873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 236/478] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 13 19:35:51 2022] POST /identity/v3/roles => generated 144 bytes in 300 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:51.668032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f50e6d-cde8-4845-b7bf-1b647b59c635 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:51.844672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f50e6d-cde8-4845-b7bf-1b647b59c635 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:51.914354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b042ef-172b-4ab2-b56d-e77a2ceec624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:51.915287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b042ef-172b-4ab2-b56d-e77a2ceec624 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:51.915711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b042ef-172b-4ab2-b56d-e77a2ceec624 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:51.916130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b042ef-172b-4ab2-b56d-e77a2ceec624 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:51.934718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b042ef-172b-4ab2-b56d-e77a2ceec624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-c3b042ef-172b-4ab2-b56d-e77a2ceec624 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:51.945674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:51.946820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:51.947734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:51.948356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 243/479] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:51.958867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:52.006576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:52.095284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26027664-f80f-4ed0-a487-05c425e86127 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:52.160190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882ed8a-d187-4666-b810-62b975977de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:52.161231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882ed8a-d187-4666-b810-62b975977de6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:52.161755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882ed8a-d187-4666-b810-62b975977de6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:52.162241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882ed8a-d187-4666-b810-62b975977de6 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:52.163625 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882ed8a-d187-4666-b810-62b975977de6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:52.164252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882ed8a-d187-4666-b810-62b975977de6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:52.168772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882ed8a-d187-4666-b810-62b975977de6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:52.183624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 237/480] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:53.422219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a640-45dd-4326-bcf3-81c039ad3c8f None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:53.422880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a640-45dd-4326-bcf3-81c039ad3c8f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:53.422880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631a640-45dd-4326-bcf3-81c039ad3c8f None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:53.424369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 244/481] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:53.429786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6317f3-5c53-4f5c-9b81-b7ac63d27d40 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:53.430385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6317f3-5c53-4f5c-9b81-b7ac63d27d40 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:53.430385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6317f3-5c53-4f5c-9b81-b7ac63d27d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:53.459624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-9c6317f3-5c53-4f5c-9b81-b7ac63d27d40 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:53.460840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9c6317f3-5c53-4f5c-9b81-b7ac63d27d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:53.552581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 238/482] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:53.979940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc498aa-c9e3-4c04-b91a-1ccc0d884a00 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:53.980315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc498aa-c9e3-4c04-b91a-1ccc0d884a00 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:53.980491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc498aa-c9e3-4c04-b91a-1ccc0d884a00 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:53.982274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 245/483] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:53.987947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d73793-c27d-413e-a298-9c6af99ae17d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:53.988240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d73793-c27d-413e-a298-9c6af99ae17d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:53.988515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d73793-c27d-413e-a298-9c6af99ae17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.008282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-09d73793-c27d-413e-a298-9c6af99ae17d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:54.009475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-09d73793-c27d-413e-a298-9c6af99ae17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:54.044920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:54.064435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc498aa-c9e3-4c04-b91a-1ccc0d884a00 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:54.115419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:54.161616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 239/484] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:54.191551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54e4c8-15fb-425b-b3ce-7626998487ad None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.191947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54e4c8-15fb-425b-b3ce-7626998487ad None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.192326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc54e4c8-15fb-425b-b3ce-7626998487ad None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.194665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 240/485] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:54.201281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60605f0-bf09-431d-8a6c-1ed645832589 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.202074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60605f0-bf09-431d-8a6c-1ed645832589 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.202732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60605f0-bf09-431d-8a6c-1ed645832589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.245998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc498aa-c9e3-4c04-b91a-1ccc0d884a00 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:54.254687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c60605f0-bf09-431d-8a6c-1ed645832589 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:54.256485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c60605f0-bf09-431d-8a6c-1ed645832589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:54.284734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf54f0-1a17-48f7-81fd-5d49ce8e9e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:54.292615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf54f0-1a17-48f7-81fd-5d49ce8e9e23 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.293202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf54f0-1a17-48f7-81fd-5d49ce8e9e23 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.293729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf54f0-1a17-48f7-81fd-5d49ce8e9e23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.307147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf54f0-1a17-48f7-81fd-5d49ce8e9e23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-55bf54f0-1a17-48f7-81fd-5d49ce8e9e23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:54.312859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:54.314134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:35:54.315055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:54.317569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 246/486] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:35:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:54.320452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dcf8-e315-4e3c-bb07-3798bb3e9f5c None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.320891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dcf8-e315-4e3c-bb07-3798bb3e9f5c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.321730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7dcf8-e315-4e3c-bb07-3798bb3e9f5c None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.324226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 247/487] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:54.330501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7dcf8-e315-4e3c-bb07-3798bb3e9f5c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:54.367207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 241/488] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:54.369544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7c292-52a5-4142-8ff8-e6c296d46762 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.369856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7c292-52a5-4142-8ff8-e6c296d46762 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.370132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a7c292-52a5-4142-8ff8-e6c296d46762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.395015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c6a7c292-52a5-4142-8ff8-e6c296d46762 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:54.397315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c6a7c292-52a5-4142-8ff8-e6c296d46762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:54.437630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:54.495344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 242/489] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:54.509307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7c292-52a5-4142-8ff8-e6c296d46762 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:54.524186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7dcf8-e315-4e3c-bb07-3798bb3e9f5c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:54.538015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:54.633395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a7c292-52a5-4142-8ff8-e6c296d46762 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:54.642496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:54.643132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.643411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.643739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.644429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] RBAC: Adding query filter params (name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:54.644782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:54.646228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:54.663874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 248/490] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:35:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:54.684042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:54.695810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38637-e476-4c7c-80a5-6b0479b5cd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:54.697337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38637-e476-4c7c-80a5-6b0479b5cd70 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.698033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38637-e476-4c7c-80a5-6b0479b5cd70 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.699950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38637-e476-4c7c-80a5-6b0479b5cd70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.718344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec38637-e476-4c7c-80a5-6b0479b5cd70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-fec38637-e476-4c7c-80a5-6b0479b5cd70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:54.726319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:54.727662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:35:54.729501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:54.733300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 243/491] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:35:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:54.735034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38637-e476-4c7c-80a5-6b0479b5cd70 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:54.813095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90340367-8b0e-4008-adf9-9c46b9faba9e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:54.854990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:54.861532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:54.862188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.862466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.862737 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.872676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:54.875078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:54.876999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:54.881217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38637-e476-4c7c-80a5-6b0479b5cd70 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:54.894047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:54.895062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:54.895922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:54.895922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:54.895922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:54.895922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 249/492] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:54.896133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:54.922026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd74fd9-2cde-4be6-adfe-23d91c3e274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:54.922708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd74fd9-2cde-4be6-adfe-23d91c3e274f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:54.923023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd74fd9-2cde-4be6-adfe-23d91c3e274f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:54.923329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd74fd9-2cde-4be6-adfe-23d91c3e274f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:54.936134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd74fd9-2cde-4be6-adfe-23d91c3e274f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-9bd74fd9-2cde-4be6-adfe-23d91c3e274f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:54.941418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:54.942434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 244/493] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:35:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:54.943854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd74fd9-2cde-4be6-adfe-23d91c3e274f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.030995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c90ab-8850-41a5-8c46-80ac87af9936 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.066362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd74fd9-2cde-4be6-adfe-23d91c3e274f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.083926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.084960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.085461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.085996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.087150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] RBAC: Adding query filter params (name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:55.087675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.090688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.101111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 250/494] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:35:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:55.102844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.115525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040319e-ee69-4348-adda-7cff87d35fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.116211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040319e-ee69-4348-adda-7cff87d35fe4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.116498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040319e-ee69-4348-adda-7cff87d35fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.116778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040319e-ee69-4348-adda-7cff87d35fe4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.125573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:55.128637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040319e-ee69-4348-adda-7cff87d35fe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-0040319e-ee69-4348-adda-7cff87d35fe4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:55.134729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:55.136428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:35:55.137465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:55.140134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 245/495] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:35:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:55.141012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040319e-ee69-4348-adda-7cff87d35fe4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.183674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:55.238298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c391e-4022-4b4c-ab3e-e237c21e4abb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.260954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040319e-ee69-4348-adda-7cff87d35fe4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.284820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.285440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.285755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.286030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.286677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:55.286993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.288367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.300773 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.305734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 251/496] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:55.305864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.305864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.305864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.306078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.306146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:55.306146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.306146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.313557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 246/497] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:55.319046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.385665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:55.427376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d13d28-85ad-4b3c-b180-7a85df94fee7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.436518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5461f8-ba88-4556-b6cd-5df66d930b3e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.450698 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:55.462072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.462676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.463532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.463783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.464443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:55.464740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.466177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.471827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.472484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.472768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.473092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.474028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 247/498] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:55.475026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.484720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.486211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:55.501830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:55.502866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:55.503720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:55.503720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:55.503720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:55.503720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 252/499] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:55.503720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.576084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8e0f1-a70e-4f2e-8428-9c7ad4c427fb None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.598949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:55.602128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b397b6-3a83-4436-b8bf-70710babaa0d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.610308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17af3b-3d36-4ea7-8971-64657d19b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.610928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17af3b-3d36-4ea7-8971-64657d19b90f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.611205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17af3b-3d36-4ea7-8971-64657d19b90f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.611477 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17af3b-3d36-4ea7-8971-64657d19b90f None admin] PATH_INFO: `/v3/users/glance` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.618660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17af3b-3d36-4ea7-8971-64657d19b90f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.620125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17af3b-3d36-4ea7-8971-64657d19b90f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-2f17af3b-3d36-4ea7-8971-64657d19b90f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:35:55.632354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:55.633462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:55.634455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 248/500] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:55.635056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b5d6d-e8e7-4ba5-a3bf-57349cf1c083 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.635056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b5d6d-e8e7-4ba5-a3bf-57349cf1c083 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.635260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b5d6d-e8e7-4ba5-a3bf-57349cf1c083 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.636428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 249/501] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:35:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:55.637112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b5d6d-e8e7-4ba5-a3bf-57349cf1c083 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.643347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.643988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.644267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.644539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.656333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.657809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:55.672723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:55.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:55.674984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:55.674984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:55.674984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:55.674984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 253/502] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:55.674984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.695282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:55.743243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b5d6d-e8e7-4ba5-a3bf-57349cf1c083 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.776564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.777191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.777469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.777775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.780279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa8205-575e-485c-88a4-9f937512a085 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.788210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.789717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:55.810410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:55.812384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:55.814402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:55.814402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:55.814402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:55.814402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 250/503] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:55.814402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.824381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2528f7-6421-4fe2-8b9a-c762b4974f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.824989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2528f7-6421-4fe2-8b9a-c762b4974f25 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.825265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2528f7-6421-4fe2-8b9a-c762b4974f25 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.825534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2528f7-6421-4fe2-8b9a-c762b4974f25 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.826307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2528f7-6421-4fe2-8b9a-c762b4974f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:55.826531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2528f7-6421-4fe2-8b9a-c762b4974f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.827900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2528f7-6421-4fe2-8b9a-c762b4974f25 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.841021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 254/504] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:55.842313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b349-6fdf-42ce-807f-5bec6b8f6b09 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.842592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b349-6fdf-42ce-807f-5bec6b8f6b09 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.842866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047b349-6fdf-42ce-807f-5bec6b8f6b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.860855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:55.864363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-b047b349-6fdf-42ce-807f-5bec6b8f6b09 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:55.865818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b047b349-6fdf-42ce-807f-5bec6b8f6b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:55.933321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28bed-5d16-46a9-9aea-a6b26640eec8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:55.960250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 255/505] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:55.961204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047b349-6fdf-42ce-807f-5bec6b8f6b09 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:55.970371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:55.971064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:55.971417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:55.971768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:55.972601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:55.972977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:55.974521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:55.992823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.021192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 251/506] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:56.022128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.112155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047b349-6fdf-42ce-807f-5bec6b8f6b09 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.141467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.148506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa122dd-f4fc-4a2c-9836-c8ad7109395c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.151269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.151893 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.152169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.152435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.153077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:56.153376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.154832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.171765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 256/507] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:56.171765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.190845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.191463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.191742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.192010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.192663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:56.192963 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.194377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.207949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 252/508] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:56.212967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.247270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.282804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbff74-0b6a-4dcd-855b-9e40b706eec9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.316252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f2c4a-532a-4737-99c0-e15d0a861985 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.316853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.317489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.317796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.318069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.326874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.328329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:35:56.340956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:56.341941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:56.342854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 257/509] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:56.343416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.356924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e281b70-da27-4e4f-9d4c-7a772a5ca267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.357534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e281b70-da27-4e4f-9d4c-7a772a5ca267 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.357847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e281b70-da27-4e4f-9d4c-7a772a5ca267 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.358117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e281b70-da27-4e4f-9d4c-7a772a5ca267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.383495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.428872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e281b70-da27-4e4f-9d4c-7a772a5ca267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.430394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e281b70-da27-4e4f-9d4c-7a772a5ca267 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.448364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990904d-56ba-4009-a36a-b6abb22aeb85 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.474228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.482841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.483471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.483749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.484020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.492166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.493645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:35:56.505591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:56.506575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 258/510] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:56.507524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.535280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 253/511] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:56.536128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e281b70-da27-4e4f-9d4c-7a772a5ca267 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.590697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.608718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc26eef-c3e9-4a0a-8bae-ed5b8da28d52 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.635665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e281b70-da27-4e4f-9d4c-7a772a5ca267 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.643099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.643727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.644000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.644267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.651974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.653410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:35:56.664619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:56.670638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 259/512] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:56.671647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.675463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.676097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.676372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.676640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.687632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.689122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:56.709348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:56.710380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:56.711223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:56.711223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:56.711223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:56.711386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 254/513] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:56.712688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.722734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.773094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb46170-1eab-40f5-8d24-028412a95afd None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.806882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.808157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.808438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.808711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.809443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:56.809782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.811387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.822068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6267dc-8e2b-4e20-bbb1-e8ac651e767c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.830228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.852780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 260/514] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:56.854193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.871908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.872523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.872797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.873066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.873827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:56.874134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:56.875898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:56.895826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 255/515] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:56.896895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:56.960118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b613c4-c74e-4309-9a93-e5fba180e5a1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:56.968117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:56.995285 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:56.996302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.996577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.996848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.002417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371b19b-fbf6-49be-ab5f-d951567b61e4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.009189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.011122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:57.028736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:57.029758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:57.030620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:57.030620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:57.030620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:57.030620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 261/516] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:57.030833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.044422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.049092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.050658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.050970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.051726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:57.052044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.053516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.072556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 256/517] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Sep 13 19:35:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:57.073654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.099113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.143928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af48fb-69a1-4813-b10c-e4a599fff217 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.180748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.181270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.181404 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c37fb-fe14-4395-a679-1a0d68e6dcf2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.181543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.181875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.182528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:57.182846 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.184222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.196688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 262/518] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:57.197587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.219462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.236843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12431de9-d123-4f58-9a3b-37ce772bba85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.237456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431de9-d123-4f58-9a3b-37ce772bba85 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.237777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431de9-d123-4f58-9a3b-37ce772bba85 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.238056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12431de9-d123-4f58-9a3b-37ce772bba85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.311753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12431de9-d123-4f58-9a3b-37ce772bba85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.313204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12431de9-d123-4f58-9a3b-37ce772bba85 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.314263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f2d76-ab6e-4bd3-96d5-64741501d3bb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.340946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.351620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ad9b4-d0fa-4a8c-9283-3398789e301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.352243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ad9b4-d0fa-4a8c-9283-3398789e301b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.352519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ad9b4-d0fa-4a8c-9283-3398789e301b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.352788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ad9b4-d0fa-4a8c-9283-3398789e301b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.425652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 257/519] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:57.425652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12431de9-d123-4f58-9a3b-37ce772bba85 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.439020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ad9b4-d0fa-4a8c-9283-3398789e301b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.440437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ad9b4-d0fa-4a8c-9283-3398789e301b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.461184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.535070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12431de9-d123-4f58-9a3b-37ce772bba85 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.545964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 263/520] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:57.546841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ad9b4-d0fa-4a8c-9283-3398789e301b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.562120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.575353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.575984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.576243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.576514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.577161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:57.577475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.578901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.591446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 258/521] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:57.592276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.664665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ad9b4-d0fa-4a8c-9283-3398789e301b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.685205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.694475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed8882-6338-4db7-90d0-eff402f14144 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.706723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b23fab-a19a-4917-a79b-6cb609340ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.707334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b23fab-a19a-4917-a79b-6cb609340ed7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.707614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b23fab-a19a-4917-a79b-6cb609340ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.707883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b23fab-a19a-4917-a79b-6cb609340ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.728483 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8bfa3-de98-4f5d-a412-2964ff6b7cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.729122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bfa3-de98-4f5d-a412-2964ff6b7cef None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.729399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bfa3-de98-4f5d-a412-2964ff6b7cef None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.729698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8bfa3-de98-4f5d-a412-2964ff6b7cef None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.730338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8bfa3-de98-4f5d-a412-2964ff6b7cef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.731790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8bfa3-de98-4f5d-a412-2964ff6b7cef None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.753528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 259/522] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:57 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:57.754463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8bfa3-de98-4f5d-a412-2964ff6b7cef None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.793780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b23fab-a19a-4917-a79b-6cb609340ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.795852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b23fab-a19a-4917-a79b-6cb609340ed7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.809927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.901070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8bfa3-de98-4f5d-a412-2964ff6b7cef None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.945497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.947072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.947974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.948263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.951691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 264/523] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:57.956984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b23fab-a19a-4917-a79b-6cb609340ed7 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.962644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.964173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:57.983387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:57.984882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:57.987225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:57.987225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:57.987225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:57.987225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 260/524] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:57.987225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.992986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.087235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b23fab-a19a-4917-a79b-6cb609340ed7 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.121943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.127292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.128056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.128326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.128600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.133434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6131710-4317-40ee-b6db-c9c5b0c3df0c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.140169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.141637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:58.157884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:58.159132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:58.159997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:58.159997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:58.159997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:58.159997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 265/525] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:58.160304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.175600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.176484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.176767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.177107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.185340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.187307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:35:58.201352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:58.202597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:58.203681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:35:58.203681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:35:58.203681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:35:58.203681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:35:58.203681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:35:58.203681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:35:58.203681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:35:58.203681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:58.206909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 261/526] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:58.208208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.242656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.285252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6ccaa-5fac-4bd1-b9ee-943e4ddf10fc None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.324053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.324699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.324980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.325255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.327528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d9227-f094-469b-badd-6ac1ce0b015b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.336391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.337928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:58.353288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:58.354837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:58.355706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:58.355706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:58.355706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:58.355706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 266/527] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:58.356539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.368058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.368677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.368961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.369303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.370477 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:58.370792 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.372204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.389457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 262/528] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.389457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.402143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.474417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054215d0-b1c1-483e-88c7-ceb3deef0663 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.498416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.509459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e6da0-f5e1-4938-be3c-71818f1ebb02 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.514853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.515474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.515808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.516085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.516738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:58.517077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.518677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.537771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 267/529] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.539161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.562405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.563313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.563788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.564265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.565447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:58.566008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.568539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.599135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 263/530] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.600619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.683950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.686092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d600ae-90ed-49d6-b263-b8ac3b826ef8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.734224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e6460-369c-4567-829c-1c96fbd2c9be None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.739201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.739969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.740375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.740777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.741558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:58.742035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.743550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.757027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 268/531] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.758590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.774744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.775378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.775657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.775919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.783320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.784809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:35:58.796436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:58.797313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 264/532] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:58.803304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.815713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.885524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce81c25-b0e9-403b-88bb-17a23c6ae103 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.918840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0393d-47fa-4a41-a537-ece5116c2446 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.922913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.922913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.923118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.923375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.930551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.932143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:35:58.945308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:58.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:58.947942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 269/533] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:58.949216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.958756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.966985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.967697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.968059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.968390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.969188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=17d78c1e1b6e46e3ba6271e6743d3c73) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:58.969585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.971132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.976859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 265/534] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/role_assignments?user.id=17d78c1e1b6e46e3ba6271e6743d3c73&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.977675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.100035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3b296-2a89-4da3-bd64-d79de4c1b659 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.101961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.118338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2fade-2fa6-48da-82d5-443407b2e895 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.145959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.146719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.147071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.147418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.156418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.158103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.166621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.167886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.168381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.168826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.169745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:59.170230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.171959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:35:59.173410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:59.174277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:35:59.175122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 270/535] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:59.176527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.192351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 266/536] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.193499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.271744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.327959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ce70a4-833b-4cf5-95e5-dcd3d3e55e02 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.335451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4077-d0dc-41d2-93d6-8fe5389d3dbe None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.371723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.371959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.372348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.372634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.372911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.373329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.373638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:59.373957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.374145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.374596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.375410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.375636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:59.376106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.377738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.389839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 267/537] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.391247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.408030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 271/538] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.409881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.454781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.542328 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c95f09-dc97-48a5-a362-fe2c3b945b53 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.552604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8b400-1412-40ea-af60-9eec4d98af0e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.557726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.595043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.596284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.596870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.597364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.598472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=2981bb0558d34329a061a5f08149bba3) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:59.598927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beac67b-a917-4096-9842-48e09537ba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.599181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.599632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beac67b-a917-4096-9842-48e09537ba28 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.599988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beac67b-a917-4096-9842-48e09537ba28 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.600335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beac67b-a917-4096-9842-48e09537ba28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.601143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beac67b-a917-4096-9842-48e09537ba28 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=9f77e57b2d96461c88e2fc33db22e22d) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:59.601523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beac67b-a917-4096-9842-48e09537ba28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.602118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.603142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beac67b-a917-4096-9842-48e09537ba28 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.610686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 268/539] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/role_assignments?user.id=2981bb0558d34329a061a5f08149bba3&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.611786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.618170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 272/540] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/role_assignments?user.id=9f77e57b2d96461c88e2fc33db22e22d&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.766248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b292f-2d54-41fa-8b15-d301c20c970a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.829160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11114467-0c74-4403-a865-6ee69726d64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.830151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114467-0c74-4403-a865-6ee69726d64c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.830622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114467-0c74-4403-a865-6ee69726d64c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.831084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114467-0c74-4403-a865-6ee69726d64c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.832058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11114467-0c74-4403-a865-6ee69726d64c None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=284740de651e4065845e945556a05ff4) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:59.832562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11114467-0c74-4403-a865-6ee69726d64c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.834352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11114467-0c74-4403-a865-6ee69726d64c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.846770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 269/541] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/role_assignments?user.id=284740de651e4065845e945556a05ff4&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:00.169367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc1067-923d-47c7-81a4-96f0829fc38a None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:00.169930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc1067-923d-47c7-81a4-96f0829fc38a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:00.169930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc1067-923d-47c7-81a4-96f0829fc38a None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:00.171291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 273/542] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:00.183842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f417dc9-b09a-421b-89c8-b14c082b0413 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:00.184169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f417dc9-b09a-421b-89c8-b14c082b0413 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:00.184462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f417dc9-b09a-421b-89c8-b14c082b0413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:00.208623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-0f417dc9-b09a-421b-89c8-b14c082b0413 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:00.209880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0f417dc9-b09a-421b-89c8-b14c082b0413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:00.256921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:00.305350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 270/543] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:00.772754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc1067-923d-47c7-81a4-96f0829fc38a None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:00.879576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:00.901034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc1067-923d-47c7-81a4-96f0829fc38a None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:00.949376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d46096-8ae9-4e87-a688-d80dd50f73b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:00.950034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46096-8ae9-4e87-a688-d80dd50f73b1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:00.950309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46096-8ae9-4e87-a688-d80dd50f73b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:00.950579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46096-8ae9-4e87-a688-d80dd50f73b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:00.966247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d46096-8ae9-4e87-a688-d80dd50f73b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-46d46096-8ae9-4e87-a688-d80dd50f73b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:00.971188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:00.972101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:00.972908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 274/544] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:00.978828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f417dc9-b09a-421b-89c8-b14c082b0413 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:01.140361 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f417dc9-b09a-421b-89c8-b14c082b0413 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:01.195019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:01.195638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:01.196169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:01.196436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:01.197136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:01.197503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:01.199137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:01.214491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 271/545] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:01.220097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d46096-8ae9-4e87-a688-d80dd50f73b1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:01.369883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d46096-8ae9-4e87-a688-d80dd50f73b1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:01.415441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:01.416555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:01.416828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:01.417095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:01.433999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:01.435466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:01.442666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:01.461301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:01.462339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:01.463771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:01.463771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:01.463771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:01.463771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 275/546] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:01.469303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:01.516936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:01.609977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae9d9b-6070-424e-a652-9367c8c89de0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:01.657830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:01.658443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:01.658722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:01.658984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:01.659635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:01.659934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:01.661302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:01.677016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 272/547] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:01.682841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:01.816046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6c3b4-a794-4ca5-a8cd-6f422f76b7bb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:01.864127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c4dbd6-0b42-4ac6-a8cd-7ed2a337e18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:01.864762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4dbd6-0b42-4ac6-a8cd-7ed2a337e18f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:01.865044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4dbd6-0b42-4ac6-a8cd-7ed2a337e18f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:01.865311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c4dbd6-0b42-4ac6-a8cd-7ed2a337e18f None admin] PATH_INFO: `/v3/users/swift` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:01.877535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4dbd6-0b42-4ac6-a8cd-7ed2a337e18f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:01.879014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c4dbd6-0b42-4ac6-a8cd-7ed2a337e18f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-04c4dbd6-0b42-4ac6-a8cd-7ed2a337e18f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:01.897407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:01.898913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:01.899985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 276/548] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:01.904889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:02.042024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:02.047067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64bc2c7-8fb1-4ae5-9c2c-01973530dc55 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:02.082007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a241-00f6-4a5b-aeb9-7c6d92a50d48 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.082353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a241-00f6-4a5b-aeb9-7c6d92a50d48 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.082530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a241-00f6-4a5b-aeb9-7c6d92a50d48 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.083854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 277/549] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:02.089237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f939-5c4e-41af-befa-b229c22b7bfe None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.089530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f939-5c4e-41af-befa-b229c22b7bfe None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.089811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0f939-5c4e-41af-befa-b229c22b7bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.112098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-86c0f939-5c4e-41af-befa-b229c22b7bfe None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:02.114618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-86c0f939-5c4e-41af-befa-b229c22b7bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:02.127520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:02.128573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.129207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.129735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.131123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:02.132022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:02.135590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:02.199354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 273/550] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 13 19:36:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:02.206496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:02.261882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:02.280301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 278/551] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:02.281914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b664f8-0eda-4cd1-bdb6-d0910fe79a05 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.282409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b664f8-0eda-4cd1-bdb6-d0910fe79a05 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.282914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b664f8-0eda-4cd1-bdb6-d0910fe79a05 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.284359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 279/552] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:02.290624 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395ac59-f7df-498a-823b-8226e134713c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.291240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395ac59-f7df-498a-823b-8226e134713c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.291807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395ac59-f7df-498a-823b-8226e134713c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.334734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5395ac59-f7df-498a-823b-8226e134713c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:02.335996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5395ac59-f7df-498a-823b-8226e134713c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:02.411805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 280/553] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:02.445204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6735cbee-1336-48ef-83af-402f2964b141 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:02.457648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffabb8-6075-4365-a4d9-eaacaa3b4dcb None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.458198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffabb8-6075-4365-a4d9-eaacaa3b4dcb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.458700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffabb8-6075-4365-a4d9-eaacaa3b4dcb None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.460437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 281/554] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:02.466645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded79a6-be14-4481-9112-a03de5ce8dd8 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.467198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded79a6-be14-4481-9112-a03de5ce8dd8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.467623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded79a6-be14-4481-9112-a03de5ce8dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.496009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-2ded79a6-be14-4481-9112-a03de5ce8dd8 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:02.498067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2ded79a6-be14-4481-9112-a03de5ce8dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:02.514801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39c3b0-e981-4b19-a4ec-0fba6c72d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:02.515661 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39c3b0-e981-4b19-a4ec-0fba6c72d8cc None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.516091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39c3b0-e981-4b19-a4ec-0fba6c72d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.516510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39c3b0-e981-4b19-a4ec-0fba6c72d8cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.555045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:02.614740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 282/555] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:02.622575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360e7b-41fa-40a1-aa6c-9cbefdbdc902 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.623325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360e7b-41fa-40a1-aa6c-9cbefdbdc902 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.623747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360e7b-41fa-40a1-aa6c-9cbefdbdc902 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.625758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 283/556] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:02.630128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360e7b-41fa-40a1-aa6c-9cbefdbdc902 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:02.636158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39c3b0-e981-4b19-a4ec-0fba6c72d8cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:02.638470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39c3b0-e981-4b19-a4ec-0fba6c72d8cc None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:02.693324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:02.751764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 274/557] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:02.757352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360e7b-41fa-40a1-aa6c-9cbefdbdc902 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:02.757633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d563224-7765-4d3c-a3cf-d48488bba2a0 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.757918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d563224-7765-4d3c-a3cf-d48488bba2a0 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.758198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d563224-7765-4d3c-a3cf-d48488bba2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.776716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1d563224-7765-4d3c-a3cf-d48488bba2a0 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:02.777968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1d563224-7765-4d3c-a3cf-d48488bba2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:02.794416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a6f63-8305-41cd-9eb9-1040e85ede48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:02.795063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a6f63-8305-41cd-9eb9-1040e85ede48 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.795465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a6f63-8305-41cd-9eb9-1040e85ede48 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.795911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a6f63-8305-41cd-9eb9-1040e85ede48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.815599 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a6f63-8305-41cd-9eb9-1040e85ede48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-d90a6f63-8305-41cd-9eb9-1040e85ede48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:02.826014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:02.827078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:02.827933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:02.828601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:02.829503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 284/558] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:02.831030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a6f63-8305-41cd-9eb9-1040e85ede48 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:02.856850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 275/559] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:02.857532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d563224-7765-4d3c-a3cf-d48488bba2a0 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:02.973126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a6f63-8305-41cd-9eb9-1040e85ede48 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:02.984782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:02.987326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d563224-7765-4d3c-a3cf-d48488bba2a0 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.018779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c287-6b32-4a8b-bd83-3af49dec4666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.019316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c287-6b32-4a8b-bd83-3af49dec4666 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.019589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c287-6b32-4a8b-bd83-3af49dec4666 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.019857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c287-6b32-4a8b-bd83-3af49dec4666 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.030906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.031540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.031821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.032097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.032810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9c287-6b32-4a8b-bd83-3af49dec4666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-27d9c287-6b32-4a8b-bd83-3af49dec4666 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:03.038175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.039169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:03.039980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:03.042442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 285/560] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.043040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c287-6b32-4a8b-bd83-3af49dec4666 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.043737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.047298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:03.063610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.064609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:03.065551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:03.065551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:03.065551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:03.069553 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 276/561] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.070416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.112846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.159532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c287-6b32-4a8b-bd83-3af49dec4666 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.182801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c04545-4e3a-4761-9a90-949178bb5f65 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.194800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.195407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.195671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.196195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.196860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] RBAC: Adding query filter params (name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:03.197771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.199139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.207206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 286/562] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:03.207206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.219610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566338-edfd-4bad-8290-92f13b403bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.220251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566338-edfd-4bad-8290-92f13b403bd0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.220530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566338-edfd-4bad-8290-92f13b403bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.220802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566338-edfd-4bad-8290-92f13b403bd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.231292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a566338-edfd-4bad-8290-92f13b403bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-0a566338-edfd-4bad-8290-92f13b403bd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:03.236888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.238592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 277/563] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.240237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566338-edfd-4bad-8290-92f13b403bd0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.245447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.336157 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.340901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d65a9-9392-4cda-86d7-26451b953d55 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.343529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566338-edfd-4bad-8290-92f13b403bd0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.376382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.376928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.377211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.377479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.378186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:03.378491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.379128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.379735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.379884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.380013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.380283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.380920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:03.381212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.382665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.388719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 287/564] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:03.389560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.399668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 278/565] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:03.399668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.495958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.505598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb4461-a220-4add-b153-e61e179a13e5 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.510311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4465d-00f3-4d3d-a9ec-3f37162a5342 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.539186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146707c-4897-41ed-aea8-0a5e446a234e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.539809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146707c-4897-41ed-aea8-0a5e446a234e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.540087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146707c-4897-41ed-aea8-0a5e446a234e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.540361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146707c-4897-41ed-aea8-0a5e446a234e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.543496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.544118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.544401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.544674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.550282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146707c-4897-41ed-aea8-0a5e446a234e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:03.554377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-6146707c-4897-41ed-aea8-0a5e446a234e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:03.555508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.556509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:03.557371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:03.558051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.558129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 288/566] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.558129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146707c-4897-41ed-aea8-0a5e446a234e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:03.572893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.574275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:03.575164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:03.575164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:03.575164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:03.575164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 279/567] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.575164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.586753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.660747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146707c-4897-41ed-aea8-0a5e446a234e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.678223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67786028-2985-4cee-8bbe-c6ab1dfe1ea1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.696543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.697796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.698076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.698349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.699018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] RBAC: Adding query filter params (name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:03.699321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.700755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.708950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 289/568] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:03.709646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.718361 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.719001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.719338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.719636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.725345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.731294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.732755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:03.750599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.751761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:03.752741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:03.752741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:03.752741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:03.753247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 280/569] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.754255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.809131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a7465-946e-4282-bdaf-621717ec760b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.812083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.843106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.843658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.843939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.844205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.850721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.852150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.853550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae0e00-3381-45aa-b389-67075f05e288 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:03.863321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.864381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 290/570] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.865307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.892052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.892651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.892929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.893204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.893901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:03.894204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.895650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.903272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 281/571] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:03.904040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.943902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.969477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f187f-e067-4c02-a42a-a0b5e3471b15 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.002372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.002973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.003249 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.003520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.003609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9f5ce-83ee-4a43-940a-7283dad2788f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.004158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:04.004461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.006749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.027028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 291/572] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.027879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.029177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.044194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.044825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.045097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.045370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.055714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.057710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:04.071578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:04.073087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:04.074094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:04.074094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:04.074094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:04.074094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 282/573] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:04.074094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.132074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e9297-b7d2-4b4d-8181-f6f74cee9a87 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.165022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.166125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.166421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.166768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.167334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:04.167636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.168995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.170435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.170952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd2a0-1ca3-40e3-9c88-ade537b4fe39 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.180779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 292/574] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.182332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.208825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.209430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.209736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.210007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.210687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:04.210992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.212413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.223994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 283/575] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.224781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.255751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.279877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae66902-eaa6-4a66-ab7d-0d9ccfaf4c77 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.314286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.315484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.315767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.316039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.326975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.327224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d0220-2fd4-48d1-8d30-ecb27a684d3e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.328411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:04.342483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:04.343856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:04.344799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:04.344799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:04.344799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:04.344799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 293/576] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:04.344799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.363825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.364456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.364737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.365013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.372985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.374471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:04.385986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:04.386965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 284/577] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:04.387923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.394532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.461872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f8432-f4ee-450a-941e-40cd01df7e41 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.493334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.503800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.504666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.505098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.505464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.505862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee17cf-89d7-49c1-9b65-9d6b3afabfa1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.506327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:04.506641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.508472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.526239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 294/578] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.527100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.549484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.550242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.550522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.550867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.558724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.560162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:04.572101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:04.573133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 285/579] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:04.574956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.644420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e8f9c-76a9-423a-9d4c-83c1cbccdcec None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.652786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.680254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.680869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.681152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.681421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.682186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=7d93aa681e2a4427a61fe0bf472c60ec) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:04.682501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.683948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.689484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 295/580] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/role_assignments?user.id=7d93aa681e2a4427a61fe0bf472c60ec&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.690333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.712415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865e3e0-ed66-4dc1-827c-a7046e8eeca6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.751442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.752072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.752350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.752622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.753299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:04.753628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.755222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.768487 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 286/581] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.769199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.787922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.824934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1573729-1e47-4e61-a133-a7884c5b3d71 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.865232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.865874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.866157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.866436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.867183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:04.867482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.868920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.890162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a3082-8cbc-4d68-8a67-6920c34c1797 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.892288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 296/582] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.893141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.933975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.944411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.945053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.945335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.945636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.954314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.955794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:04.968926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:04.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:04.971191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 287/583] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:04.975589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.023320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d5bf1-55ab-42cb-a2c7-a011fcbba7c4 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.054106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.066693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.067520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.067901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.068294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.069128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:05.069532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.071576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.087023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c7326-5220-434e-8fe4-1f1f1069287b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.097172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 297/584] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.097172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.126667 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.127313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.127601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.127874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.136261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.137757 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:05.149443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:05.151264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 288/585] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:05.152215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.199844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.217307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29289d0b-67ca-4728-8d3d-7c69fff10c80 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.258801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.259484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.260324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.260551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.264993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c2b47-09ca-487c-aefb-5d5b51423b32 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.270744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.272194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:05.286657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:05.287828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:05.288899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:05.288899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:05.288899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:05.288899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 298/586] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:05.288899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.299681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.305694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.307075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.307624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.308043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.309032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:05.309490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.311258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.332089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 289/587] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.333014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.406871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a38ee-c822-4592-beda-6208b3a465f9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.440456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047828a-dee1-42cc-a454-4a0292ab1b39 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.443037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.443761 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.444111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.444450 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.452964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.458211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.459797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:05.475699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:05.476917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:05.478043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:05.478043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:05.478043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:05.481762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 299/588] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:05.482700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.487055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.487796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.488084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.488356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.489096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:05.489401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.490898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.515408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 290/589] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.515408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.565792 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.607048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1978cd6-762d-49ef-8517-f7156c45cac8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.623849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3af4dd-15d1-4580-a35e-a0a4dd8e0837 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.642697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.643303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.643575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.643839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.644540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:05.644844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.646330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.658442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 300/590] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.660719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.661420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.661797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.662135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.663655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.673730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.675258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:05.691782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:05.692958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:05.694102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:05.694102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:05.694102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:05.694761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 291/591] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:05.695605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.724882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.783544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a941d7c-ea8e-4d16-951d-9513b19d32de None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.801327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.804819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5875014-3bb5-4f57-8102-2718448e8cc4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.824615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.825527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.825984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.826265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.835445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:05.835445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.835445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.849730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 301/592] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.849730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.852092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.852715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.852998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.853833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.866003 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.867621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:05.883191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:05.884394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:05.885332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:05.885332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:05.885332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:05.885332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 292/593] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:05.885567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.976287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3dfaf-0e31-41d2-820f-47e13a88d824 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.987293 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.012380 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.013014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.013294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.013571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.021660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.023369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.030331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d7de70-665a-4e54-a860-8de8eeff0abb None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:06.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:06.036608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:06.038129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:06.038129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:06.038129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:06.038129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:06.038129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:06.038129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:06.038129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:06.038129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:06.039087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 302/594] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:06.039741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.071683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.072418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.072595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.072871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.073514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:06.074003 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.075231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.088531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 293/595] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:06.093577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.109330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.154945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03009b2-2858-4db1-8007-93adc4e94dae None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.193857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.194487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.194796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.195040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.202433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.202870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67f6bb-3e1e-43ed-a590-e68ba2b7d3e6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.203880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:06.215906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:06.216809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:06.217795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:06.217795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:06.217795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:06.217795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:06.217795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:06.217795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:06.217795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:06.217795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:06.221847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 303/596] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:06.222666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.243054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.243588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.243902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.244178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.244826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:06.245132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.246544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.254503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.263480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 294/597] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:06.263480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.345562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.355359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76756058-08df-40a5-9aa3-584182c03585 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.382145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02054340-0d24-443a-9ead-91fee0c90356 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.397042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.397679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.397956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.398224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.398923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:06.399222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.401257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.419278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 304/598] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:06.419278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.426231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.427035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.427423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.427874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.438849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.441809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:06.465381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:06.467390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 295/599] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:06.469181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.531302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.544848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d0651-d09e-48cf-ba91-3ebe8a8331d8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.582670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.583890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.584504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.585148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.586472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:06.587093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.590463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.601730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6ad18-b139-4ed1-bf2a-805d191489ef None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.606937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 305/600] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:06.608023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.639763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.645154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.646378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.646870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.646988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.655321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.656777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:06.668658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:06.669594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:06.670541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 296/601] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:06.671036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.731443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95133607-468e-4e1c-bd82-86ab3aa4f69d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.772999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9a00-23ad-4241-8f91-339ab79284c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.773722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9a00-23ad-4241-8f91-339ab79284c3 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.774003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9a00-23ad-4241-8f91-339ab79284c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.774277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9a00-23ad-4241-8f91-339ab79284c3 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/17d78c1e1b6e46e3ba6271e6743d3c73/roles/af374fa10d7d4a7aa378572855842eb0` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.786968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d1896-1e40-4f27-9825-dcb571c05534 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.791354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb9a00-23ad-4241-8f91-339ab79284c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=17d78c1e1b6e46e3ba6271e6743d3c73, role_id=af374fa10d7d4a7aa378572855842eb0)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.792892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb9a00-23ad-4241-8f91-339ab79284c3 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.802510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.808490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 306/602] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:06 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/17d78c1e1b6e46e3ba6271e6743d3c73/roles/af374fa10d7d4a7aa378572855842eb0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:06.809868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9a00-23ad-4241-8f91-339ab79284c3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.837181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.838184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.838614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.839168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.839911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:06.840221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.841880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.861551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 297/603] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:06.862691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.940809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.949592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9a00-23ad-4241-8f91-339ab79284c3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.990313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ad0cb-ac5c-4c2c-89bf-5cbe6609e005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.991006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad0cb-ac5c-4c2c-89bf-5cbe6609e005 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.991283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad0cb-ac5c-4c2c-89bf-5cbe6609e005 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.991552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ad0cb-ac5c-4c2c-89bf-5cbe6609e005 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/2981bb0558d34329a061a5f08149bba3/roles/af374fa10d7d4a7aa378572855842eb0` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.008007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf152c6-e5f2-419b-a00a-7d68bfc45407 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:07.014987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ad0cb-ac5c-4c2c-89bf-5cbe6609e005 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=2981bb0558d34329a061a5f08149bba3, role_id=af374fa10d7d4a7aa378572855842eb0)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:07.017657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ad0cb-ac5c-4c2c-89bf-5cbe6609e005 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:07.034273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 307/604] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:06 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/2981bb0558d34329a061a5f08149bba3/roles/af374fa10d7d4a7aa378572855842eb0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:07.035296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ad0cb-ac5c-4c2c-89bf-5cbe6609e005 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:07.068691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aba6e2-33ca-49a4-9eba-5795f7aafc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:07.069399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba6e2-33ca-49a4-9eba-5795f7aafc2d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.069826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba6e2-33ca-49a4-9eba-5795f7aafc2d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.070534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aba6e2-33ca-49a4-9eba-5795f7aafc2d None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.072759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aba6e2-33ca-49a4-9eba-5795f7aafc2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:07.073440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aba6e2-33ca-49a4-9eba-5795f7aafc2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:07.075425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aba6e2-33ca-49a4-9eba-5795f7aafc2d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:07.094292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 298/605] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:07.096495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b5d2-dd0b-4838-8710-630d7f7f7fc3 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.097216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b5d2-dd0b-4838-8710-630d7f7f7fc3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.097638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b5d2-dd0b-4838-8710-630d7f7f7fc3 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.100225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:07.105194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 299/606] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:07.106496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b5d2-dd0b-4838-8710-630d7f7f7fc3 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:07.169215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ad0cb-ac5c-4c2c-89bf-5cbe6609e005 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:07.212301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e843fba-e766-4efb-b66b-8e3f09dddfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:07.213321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843fba-e766-4efb-b66b-8e3f09dddfa0 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.213853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843fba-e766-4efb-b66b-8e3f09dddfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.214277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e843fba-e766-4efb-b66b-8e3f09dddfa0 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/9f77e57b2d96461c88e2fc33db22e22d/roles/af374fa10d7d4a7aa378572855842eb0` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.235040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e843fba-e766-4efb-b66b-8e3f09dddfa0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=9f77e57b2d96461c88e2fc33db22e22d, role_id=af374fa10d7d4a7aa378572855842eb0)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:07.236757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e843fba-e766-4efb-b66b-8e3f09dddfa0 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:07.255041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 308/607] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:07 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/9f77e57b2d96461c88e2fc33db22e22d/roles/af374fa10d7d4a7aa378572855842eb0 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:07.256567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6d6e-837e-404f-bef2-7d417bd2b391 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.257060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6d6e-837e-404f-bef2-7d417bd2b391 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.257447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b5d2-dd0b-4838-8710-630d7f7f7fc3 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:07.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e6d6e-837e-404f-bef2-7d417bd2b391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.275632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:07.282552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-445e6d6e-837e-404f-bef2-7d417bd2b391 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:07.283786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-445e6d6e-837e-404f-bef2-7d417bd2b391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:07.305678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf25114-1a3f-451b-8d0e-995319ab3a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:07.307475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf25114-1a3f-451b-8d0e-995319ab3a1d None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.307861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf25114-1a3f-451b-8d0e-995319ab3a1d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.308238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf25114-1a3f-451b-8d0e-995319ab3a1d None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/284740de651e4065845e945556a05ff4/roles/af374fa10d7d4a7aa378572855842eb0` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.327885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf25114-1a3f-451b-8d0e-995319ab3a1d None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=284740de651e4065845e945556a05ff4, role_id=af374fa10d7d4a7aa378572855842eb0)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:07.334088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf25114-1a3f-451b-8d0e-995319ab3a1d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:07.354582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 300/608] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:07 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/284740de651e4065845e945556a05ff4/roles/af374fa10d7d4a7aa378572855842eb0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:07.403191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 309/609] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:07.755356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf25114-1a3f-451b-8d0e-995319ab3a1d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:07.881384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf25114-1a3f-451b-8d0e-995319ab3a1d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:07.933098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a631e3-0456-4d9b-af35-e6acc275b445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:07.933098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a631e3-0456-4d9b-af35-e6acc275b445 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.933098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a631e3-0456-4d9b-af35-e6acc275b445 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.933098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a631e3-0456-4d9b-af35-e6acc275b445 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.952368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a631e3-0456-4d9b-af35-e6acc275b445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-10a631e3-0456-4d9b-af35-e6acc275b445 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:07.958228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:07.959344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:07.960399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:07.962457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 301/610] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:07.968660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e6d6e-837e-404f-bef2-7d417bd2b391 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.030323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:08.128342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e6d6e-837e-404f-bef2-7d417bd2b391 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.177969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.178597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.178878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.179153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.179830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] RBAC: Adding query filter params (name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:08.180134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.181575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:08.190981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 310/611] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:08.196918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a631e3-0456-4d9b-af35-e6acc275b445 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.240824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:08.357021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a631e3-0456-4d9b-af35-e6acc275b445 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.409160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.410110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.410535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.410952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.425554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.427317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:08.449450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:08.450672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:08.451762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:08.451762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:08.451762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:08.453481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 302/612] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:08.459218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.614263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:08.630125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ad886-8ad3-47ad-98e5-398e2431fb70 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.673942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.674617 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.674953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.675231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.675889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:08.676192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.677568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:08.693639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 311/613] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:08.699756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.819653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:08.856106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9c930-d8e6-47c3-b9bd-1c31271e6fc8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.903047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.903596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.903874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.904141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.912745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.914270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:08.929685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:08.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:08.932840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 303/614] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:08.938081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.087949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8154f814-eb3a-4140-b933-29403c85cb9d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.124853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:09.125659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.126040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.126411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.127252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:09.127680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:09.129210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:09.147994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 312/615] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 13 19:36:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:09.154029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.319024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290aa984-85d1-46de-b48b-61744d3a4da1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.366435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce95d2-2c87-4d80-8ca2-c83c73a0df05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:09.367129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce95d2-2c87-4d80-8ca2-c83c73a0df05 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.367417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce95d2-2c87-4d80-8ca2-c83c73a0df05 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.367692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce95d2-2c87-4d80-8ca2-c83c73a0df05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.370029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:09.385157 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce95d2-2c87-4d80-8ca2-c83c73a0df05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:09.386693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce95d2-2c87-4d80-8ca2-c83c73a0df05 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-d6ce95d2-2c87-4d80-8ca2-c83c73a0df05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:09.405759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:09.406716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:09.407597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:09.407597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:09.407597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:09.407597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 304/616] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:09.413583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.452288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:09.614072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89be844-f4dc-4b30-99f5-fb4e33bda498 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.615875 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d22ff5-71ba-4bb0-b813-9da5437d1e04 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.616446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d22ff5-71ba-4bb0-b813-9da5437d1e04 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.616953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d22ff5-71ba-4bb0-b813-9da5437d1e04 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.621546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 305/617] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:09.624048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049dfd0-628c-4d79-8eb8-6590b4152866 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.624558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049dfd0-628c-4d79-8eb8-6590b4152866 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.625038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049dfd0-628c-4d79-8eb8-6590b4152866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.673472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8049dfd0-628c-4d79-8eb8-6590b4152866 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:09.675459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8049dfd0-628c-4d79-8eb8-6590b4152866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:09.693403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:09.694425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.694909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.695486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.696641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:09.697151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:09.700954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:09.726492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 313/618] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:09.731540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.838260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:09.867658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 306/619] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:09.870295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956feeb-a61b-4c15-b526-1b202d4267b6 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.870755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956feeb-a61b-4c15-b526-1b202d4267b6 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.871047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956feeb-a61b-4c15-b526-1b202d4267b6 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.876082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 307/620] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:09.893390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a476-8851-4fbc-8206-3f01066fcf28 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.893833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a476-8851-4fbc-8206-3f01066fcf28 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.894309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a476-8851-4fbc-8206-3f01066fcf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.934347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1ad482-a93e-4bec-aa6c-8e75135cd0b7 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.944495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-0384a476-8851-4fbc-8206-3f01066fcf28 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:09.946732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0384a476-8851-4fbc-8206-3f01066fcf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:10.033385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.034095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.034378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.034648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.049212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.050753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:10.080810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:10.081834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:10.082753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:10.082753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:10.082753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:10.082753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:10.082753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:10.082753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:10.082753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:10.082753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:10.085758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 314/621] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:10.091274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 308/622] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:10.091996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.185109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384a476-8851-4fbc-8206-3f01066fcf28 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.235800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f368d8d0-eeb7-4506-849c-1b8fac927de6 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.271356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.272700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.287100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d640199-771c-42ef-bb43-defc7af374fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.287705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d640199-771c-42ef-bb43-defc7af374fa None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.287978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d640199-771c-42ef-bb43-defc7af374fa None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.288244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d640199-771c-42ef-bb43-defc7af374fa None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.288929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d640199-771c-42ef-bb43-defc7af374fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:10.289228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d640199-771c-42ef-bb43-defc7af374fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.290723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d640199-771c-42ef-bb43-defc7af374fa None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.303593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 315/623] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:10.304782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73634-ad22-4807-9193-c6ecabd561e6 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.305060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73634-ad22-4807-9193-c6ecabd561e6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.305327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73634-ad22-4807-9193-c6ecabd561e6 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.311650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 316/624] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:10.312402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff73634-ad22-4807-9193-c6ecabd561e6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.349857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384a476-8851-4fbc-8206-3f01066fcf28 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.397516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ae71b-29ec-4107-9927-aa642df1b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.398825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ae71b-29ec-4107-9927-aa642df1b81c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.400354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ae71b-29ec-4107-9927-aa642df1b81c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.400830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ae71b-29ec-4107-9927-aa642df1b81c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.416636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ae71b-29ec-4107-9927-aa642df1b81c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-251ae71b-29ec-4107-9927-aa642df1b81c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:10.421929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:10.423137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:10.424196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:10.426608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 309/625] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:10.428079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c865b-f075-4d09-ba1c-c63efcc14c28 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.428364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c865b-f075-4d09-ba1c-c63efcc14c28 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.428641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c865b-f075-4d09-ba1c-c63efcc14c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.438074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff73634-ad22-4807-9193-c6ecabd561e6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.450603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-882c865b-f075-4d09-ba1c-c63efcc14c28 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:10.451895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-882c865b-f075-4d09-ba1c-c63efcc14c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:10.476389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2947ff-9c71-4808-b507-305af9a7a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.476986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2947ff-9c71-4808-b507-305af9a7a49d None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.477260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2947ff-9c71-4808-b507-305af9a7a49d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.477765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2947ff-9c71-4808-b507-305af9a7a49d None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/7d93aa681e2a4427a61fe0bf472c60ec/roles/af374fa10d7d4a7aa378572855842eb0` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.493255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2947ff-9c71-4808-b507-305af9a7a49d None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=7d93aa681e2a4427a61fe0bf472c60ec, role_id=af374fa10d7d4a7aa378572855842eb0)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.495284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2947ff-9c71-4808-b507-305af9a7a49d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.510019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 317/626] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:10 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/7d93aa681e2a4427a61fe0bf472c60ec/roles/af374fa10d7d4a7aa378572855842eb0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:10.510919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2947ff-9c71-4808-b507-305af9a7a49d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.535865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 310/627] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:10.543589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c865b-f075-4d09-ba1c-c63efcc14c28 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.599904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.609647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.688392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2947ff-9c71-4808-b507-305af9a7a49d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.700482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c865b-f075-4d09-ba1c-c63efcc14c28 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.732936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d7cf5-5338-4f30-8cd0-6e99f4ef9ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.733580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d7cf5-5338-4f30-8cd0-6e99f4ef9ac9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.733887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d7cf5-5338-4f30-8cd0-6e99f4ef9ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.734156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d7cf5-5338-4f30-8cd0-6e99f4ef9ac9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.740384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73f4f8-cf2c-427d-abf1-65de3950e10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.740998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73f4f8-cf2c-427d-abf1-65de3950e10d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.741274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73f4f8-cf2c-427d-abf1-65de3950e10d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.741538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73f4f8-cf2c-427d-abf1-65de3950e10d None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.745919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73f4f8-cf2c-427d-abf1-65de3950e10d None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:10.745919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73f4f8-cf2c-427d-abf1-65de3950e10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.745919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73f4f8-cf2c-427d-abf1-65de3950e10d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.749296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d7cf5-5338-4f30-8cd0-6e99f4ef9ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:10.753207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 311/628] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:10.754452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf970ee-42bf-49b0-9ed4-d5f49c314200 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-183d7cf5-5338-4f30-8cd0-6e99f4ef9ac9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:10.754561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:10.755455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:10.756309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:10.756939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf970ee-42bf-49b0-9ed4-d5f49c314200 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.756939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf970ee-42bf-49b0-9ed4-d5f49c314200 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.757062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 318/629] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:10.761728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d7cf5-5338-4f30-8cd0-6e99f4ef9ac9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.764998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 312/630] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:10.766316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf970ee-42bf-49b0-9ed4-d5f49c314200 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.872224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.875598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d7cf5-5338-4f30-8cd0-6e99f4ef9ac9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.876796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.882715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf970ee-42bf-49b0-9ed4-d5f49c314200 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.912126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.912756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.913035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.913309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.919303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b154c8-0b45-42bb-a525-db812df68ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.919916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b154c8-0b45-42bb-a525-db812df68ee9 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.920201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b154c8-0b45-42bb-a525-db812df68ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.920475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b154c8-0b45-42bb-a525-db812df68ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.921424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b154c8-0b45-42bb-a525-db812df68ee9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:10.921424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b154c8-0b45-42bb-a525-db812df68ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.922832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b154c8-0b45-42bb-a525-db812df68ee9 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.925336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.926804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.936886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 313/631] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:10.936886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8db651-2ea8-4cd2-a79c-ba0dbe5d9d84 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.936886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8db651-2ea8-4cd2-a79c-ba0dbe5d9d84 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.936886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8db651-2ea8-4cd2-a79c-ba0dbe5d9d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:10.942102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:10.943054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:10.943975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:10.943975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:10.943975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:10.948577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 319/632] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:10.949383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.957268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7f8db651-2ea8-4cd2-a79c-ba0dbe5d9d84 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:10.958486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7f8db651-2ea8-4cd2-a79c-ba0dbe5d9d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:11.036912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 314/633] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:11.038065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8db651-2ea8-4cd2-a79c-ba0dbe5d9d84 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.082135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244f15a-3c39-4fee-bf3d-c199794cecac None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.118358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07426d46-684b-41a1-b9d2-54f20dfb857a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.120621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07426d46-684b-41a1-b9d2-54f20dfb857a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.121075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07426d46-684b-41a1-b9d2-54f20dfb857a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.121520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07426d46-684b-41a1-b9d2-54f20dfb857a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.137068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07426d46-684b-41a1-b9d2-54f20dfb857a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-07426d46-684b-41a1-b9d2-54f20dfb857a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:11.143651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:11.144625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:11.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:11.148213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 320/634] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:11.148331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.150449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07426d46-684b-41a1-b9d2-54f20dfb857a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.162518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8db651-2ea8-4cd2-a79c-ba0dbe5d9d84 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.170322 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.215577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.216214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.216568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.216912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.232478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.234553 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:11.252956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:11.254876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:11.256670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:11.256670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:11.256670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:11.258939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 315/635] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:11.260769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.311608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07426d46-684b-41a1-b9d2-54f20dfb857a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.374595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.375540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.376013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.376470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.377635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:11.378067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.380369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.390136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc09684-7441-4c9d-a278-ed63f0c7789e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.401587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 321/636] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:11.403083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.442330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.443359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.443851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.444317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.445444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:11.446243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.448827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.468932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 316/637] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:11.468932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.474134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.493942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.530068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423c7f0-edeb-4022-a146-acb856559ca3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.566815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.567425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.567703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.567973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.568610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:11.568912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.570302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.588020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 322/638] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:11.588020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.592247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77b125-0f24-4c92-afed-04b4167aab4c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.626924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820b37b-21c7-4b32-9bb3-685e222169e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.627558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820b37b-21c7-4b32-9bb3-685e222169e4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.627935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820b37b-21c7-4b32-9bb3-685e222169e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.628215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820b37b-21c7-4b32-9bb3-685e222169e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.639285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f820b37b-21c7-4b32-9bb3-685e222169e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-f820b37b-21c7-4b32-9bb3-685e222169e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:11.644628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:11.645594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 317/639] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:11.647111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820b37b-21c7-4b32-9bb3-685e222169e4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.699607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3c3f5-abd3-4a70-9163-0908051a72d8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.709951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.730778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.736397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.737058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.737308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.737577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] PATH_INFO: `/v3/users/glance` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.745244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.746747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.750675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f820b37b-21c7-4b32-9bb3-685e222169e4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:11.758068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:11.759614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:11.760558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 323/640] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:11.761143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.789141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.789805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.790089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.790386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.801158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.802641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:11.820151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:11.821993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:11.822922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:11.822922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:11.822922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:11.826708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 318/641] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:11.827475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.872587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505f36d-5443-46c3-82ba-b3e324057b4b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.909022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.909670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.909945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.910213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.918039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.919465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.930081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffb02a-951f-4e3b-8f7c-f13ba86f4e99 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:11.934385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:11.935334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 324/642] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:11.941449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.949909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.970152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.970760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.971042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.971393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.972002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:11.972307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.973706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.985714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 319/643] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:11.985714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.007140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.066454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab4a34-cd22-4d35-b57b-414d65329434 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.094049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54f0f9-c9b3-48f8-941e-fc722bbd34d2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.105319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.106247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.106532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.106807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.107543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:12.107848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.129942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 325/644] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:12.130783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.131041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.131798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.132206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.132603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.133385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:12.133871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.136021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.148709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 320/645] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:12.149568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.194435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.253593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1270fdf-6a15-4097-95a0-75f3ccc681e8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.260874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd55de-3214-4303-bbc0-bc42767fd3b0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.266111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.291188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.291871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.292066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.292337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.293119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:12.293372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.294838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.297009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.297684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.297970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.298250 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.309304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.310882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.317499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 326/646] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:12.317499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:12.333880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:12.334892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:12.335790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:12.335790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:12.335790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:12.335790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 321/647] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:12.336030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.428771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dddb2d1-9393-43cd-8f75-b5c6e5d8f308 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.431586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.442363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f895fca-45d4-4cfd-8954-a17337450c25 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.470344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee06af-e2f1-4aa7-9404-72d40209cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.471061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee06af-e2f1-4aa7-9404-72d40209cc2e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.471378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee06af-e2f1-4aa7-9404-72d40209cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.471705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee06af-e2f1-4aa7-9404-72d40209cc2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.479102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.479740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.480021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.480293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.488069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.489520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:12.501601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:12.502628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 322/648] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:12.503644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.518963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.560353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee06af-e2f1-4aa7-9404-72d40209cc2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.561883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee06af-e2f1-4aa7-9404-72d40209cc2e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.616452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf3a59-fd9c-4be4-96f1-8bd43b7fe892 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.654648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1a41d-bea7-446e-8a52-e731bebd6695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.656068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a41d-bea7-446e-8a52-e731bebd6695 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.656777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a41d-bea7-446e-8a52-e731bebd6695 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.657290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1a41d-bea7-446e-8a52-e731bebd6695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.683691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.687692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 327/649] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:12.688826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee06af-e2f1-4aa7-9404-72d40209cc2e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.773200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.782024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1a41d-bea7-446e-8a52-e731bebd6695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.788050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1a41d-bea7-446e-8a52-e731bebd6695 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.825090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee06af-e2f1-4aa7-9404-72d40209cc2e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.863556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.864170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.864446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.864823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.865593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:12.865985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.867503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.880625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 328/650] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:12.881469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.902594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 323/651] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:12.903458 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1a41d-bea7-446e-8a52-e731bebd6695 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.959061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.993039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe49e55-33d4-4be9-99a1-5692f4e7dca9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.018989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1a41d-bea7-446e-8a52-e731bebd6695 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.032614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.042058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e12c9c-c176-4b3a-8bf9-2c85a47913e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.042675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e12c9c-c176-4b3a-8bf9-2c85a47913e5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.042953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e12c9c-c176-4b3a-8bf9-2c85a47913e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.043221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e12c9c-c176-4b3a-8bf9-2c85a47913e5 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.043951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e12c9c-c176-4b3a-8bf9-2c85a47913e5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:13.044253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e12c9c-c176-4b3a-8bf9-2c85a47913e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.045710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e12c9c-c176-4b3a-8bf9-2c85a47913e5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.057926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.058556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.058836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.059105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.064965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 329/652] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:13.066253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e4689-c2de-4f6e-806e-c6ca50664576 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.066568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e4689-c2de-4f6e-806e-c6ca50664576 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.066842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e4689-c2de-4f6e-806e-c6ca50664576 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.068076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 330/653] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:13.068783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e4689-c2de-4f6e-806e-c6ca50664576 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.076108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.077713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:13.093466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:13.094756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:13.095820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:13.095820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:13.095820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:13.095820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 324/654] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:13.095820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.175552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e4689-c2de-4f6e-806e-c6ca50664576 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.194028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267558d-df54-46d8-b594-1dcf8b633d39 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.195530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.211676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.212883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.213163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.213435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.220516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.221998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.228724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12dc3e9-807d-442a-8ffe-721b877c9258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.229344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc3e9-807d-442a-8ffe-721b877c9258 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.229653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc3e9-807d-442a-8ffe-721b877c9258 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.229927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dc3e9-807d-442a-8ffe-721b877c9258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:13.233640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:13.234760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 331/655] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:13.235742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.240505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12dc3e9-807d-442a-8ffe-721b877c9258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.242030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12dc3e9-807d-442a-8ffe-721b877c9258 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-e12dc3e9-807d-442a-8ffe-721b877c9258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:13.256719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:13.257979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:13.259475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:13.259475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:13.259475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:13.259475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 325/656] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:13.259713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1ea12-7300-4d40-8066-5f58501f5727 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.259835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1ea12-7300-4d40-8066-5f58501f5727 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.260114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1ea12-7300-4d40-8066-5f58501f5727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.278624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ccb1ea12-7300-4d40-8066-5f58501f5727 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:13.279836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ccb1ea12-7300-4d40-8066-5f58501f5727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:13.291475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.344419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5ca7a-c6d4-49f1-8cbd-04aa1aa1bfbc None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.346826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 326/657] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:13.347675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1ea12-7300-4d40-8066-5f58501f5727 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.386950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb5d62-dc54-4d07-83e1-d5e06137a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.387578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb5d62-dc54-4d07-83e1-d5e06137a5be None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.387899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb5d62-dc54-4d07-83e1-d5e06137a5be None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.388382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb5d62-dc54-4d07-83e1-d5e06137a5be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.473135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.485381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb5d62-dc54-4d07-83e1-d5e06137a5be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.488216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb5d62-dc54-4d07-83e1-d5e06137a5be None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.495331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1ea12-7300-4d40-8066-5f58501f5727 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.534898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.535908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.536887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.537312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.538849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:13.539408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.541629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.542044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.560125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 327/658] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:13.560983 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.610965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 332/659] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:13.610965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb5d62-dc54-4d07-83e1-d5e06137a5be None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.699268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae48e4-aa7a-4590-97c6-add49e6ee62a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.719809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb5d62-dc54-4d07-83e1-d5e06137a5be None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.732085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.732701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.732983 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.733265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.734030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:13.734340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.735795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.753309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.753943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.754058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 328/660] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:13.754241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.754493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.754888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.755138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:13.755440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.757034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.759924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.770086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 333/661] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:13.770782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.779660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.870469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c1292-6d4a-442e-b061-66c14165fafb None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.874698 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41d78f-0ba0-41aa-b56b-4350a9ad9bca None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.906597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.907239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.907518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.907884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.910157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.910785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.911056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.911321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.914943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.916965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.923268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.924699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:13.929420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:13.930691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 329/662] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:13.933484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:13.945219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:13.947842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:13.950186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:13.950186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:13.950186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:13.950186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 334/663] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:13.950186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.012944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.016462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.060406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ba96a-e60e-4290-869a-8ccb6db7c8fd None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.060690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b597f-381d-4ed1-b1e2-f49c11fb5d17 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.096053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9788e83-de32-424a-b250-4ccc220315c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.096661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9788e83-de32-424a-b250-4ccc220315c3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.096940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9788e83-de32-424a-b250-4ccc220315c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.097209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9788e83-de32-424a-b250-4ccc220315c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.102403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd037b-a239-4024-b833-4ad7486be70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.103542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd037b-a239-4024-b833-4ad7486be70c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.103824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd037b-a239-4024-b833-4ad7486be70c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.104094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd037b-a239-4024-b833-4ad7486be70c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.114876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd037b-a239-4024-b833-4ad7486be70c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-acdd037b-a239-4024-b833-4ad7486be70c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:14.120650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:14.121912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 330/664] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:14.123079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd037b-a239-4024-b833-4ad7486be70c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.179559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9788e83-de32-424a-b250-4ccc220315c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.183128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9788e83-de32-424a-b250-4ccc220315c3 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.233247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd037b-a239-4024-b833-4ad7486be70c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.245398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.255765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.267087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.267714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.267989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.268258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.275245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.276690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:14.287845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:14.293460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:14.294584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:14.294584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:14.294584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:14.294584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:14.294584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:14.294584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:14.294584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:14.294584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:14.295112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 335/665] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:14.295206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 331/666] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:14.295295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9788e83-de32-424a-b250-4ccc220315c3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.295382 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.392946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9788e83-de32-424a-b250-4ccc220315c3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.393424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ba1b1-51c9-48fd-843c-776484958039 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.426025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.426351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.426627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.426909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.427022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.427185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.427282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.427457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.427882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.428138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.428235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.428437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.429558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.429865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.442314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 336/668] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.442484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 332/668] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.442973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.443933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.471249 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.489579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.548106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b53bb-f7ca-405b-9830-d1c4d9965ad2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.556281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53b8b7-1ae2-4432-a654-5c23a44560fb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.582627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.583258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.583543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.583819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.590214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.590815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.591091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.591361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.592028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] RBAC: Adding query filter params (name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.592329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.593728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.593887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.595346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.601732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 337/669] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.602526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:14.615493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:14.616666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:14.617996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:14.617996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:14.617996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:14.617996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 333/670] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:14.617996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.688431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.712829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25500d-c8b8-4f83-9dc7-e52a7b4cb298 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.718255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.725558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca5e03-80c9-468b-90af-651a8f3716cf None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.749001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.749634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.749923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.750206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.750891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.751198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.752627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.760970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.762128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.762412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.762683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.764823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 338/671] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.765728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.773790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.776598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:14.788053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:14.789089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 334/672] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:14.790817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.868652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082cdce0-235b-4eb3-972a-2ee5c47dcfe1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.892502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0761a7-6741-4025-9d44-0fd7e32c9ad6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.902193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.902796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.903068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.903333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.904050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=17d78c1e1b6e46e3ba6271e6743d3c73) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.904349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.905812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.911457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 339/673] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/role_assignments?user.id=17d78c1e1b6e46e3ba6271e6743d3c73&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.912222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.916085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.949597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.950358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.950743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.951123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.963216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.964829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:14.981311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:14.982601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:14.983714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:14.983714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:14.983714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:14.984141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 335/674] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:14.989765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.001933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.037950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3ab71-1214-485f-9a35-a1309f69f5b5 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.075834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.076561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.076906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.077247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.078011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.078382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.079850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.094710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 340/675] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.095735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.116289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e12568-b56a-4af2-af0a-34b8217c2554 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.155823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.156437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.156724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.157003 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.157851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=2981bb0558d34329a061a5f08149bba3) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.158169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.159695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.166126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.167234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 336/676] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/role_assignments?user.id=2981bb0558d34329a061a5f08149bba3&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.167772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.222597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c652e7-8ddb-4def-abf1-f15cf548f752 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.263849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.272379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.273568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.274001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.274370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.275710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.276118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.277837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.297724 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 341/677] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.297724 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.315528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2e23b-0960-45af-aef7-e7ee6bc25739 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.364252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.365285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.365741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.366171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.366961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.367419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.369446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.383911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 337/678] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.385176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.432850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea76f5-8334-435d-a978-d63dcea34f16 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.462113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.480812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.480812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.480812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.481179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.489781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.491265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.505068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eda57-db2c-4438-9fff-f820e48489f6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:15.505590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:15.506465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:15.507346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 342/679] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:15.507849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.548546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.560899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.561940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.562263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.562824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.563686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=9f77e57b2d96461c88e2fc33db22e22d) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.564006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.566045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.574592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 338/680] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/role_assignments?user.id=9f77e57b2d96461c88e2fc33db22e22d&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.575754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.631232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552ae09-a20b-4748-af8e-2a35580c9104 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.677090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.677816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.678102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.678379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.689306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.691581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.709297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d500c56-67ee-42ad-a5e6-e553b69ad5f5 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:15.709576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:15.711311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:15.713071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 343/681] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:15.718269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.806350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.806897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.807172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.807439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.808142 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.808445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.809919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.827287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 339/682] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.832245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.833833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.904444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cc697b-e463-490c-949d-d12885c48146 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.907431 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.960387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.961313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.962142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.963103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.964395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.964886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.966787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.969002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3c967-64fd-449f-944e-15ebe333dc6c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.997875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 344/683] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:16.004033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:16.030715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:16.031862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.032345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.032940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.033991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=284740de651e4065845e945556a05ff4) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:16.034472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:16.036067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:16.050441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 340/684] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/role_assignments?user.id=284740de651e4065845e945556a05ff4&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:16.185810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dadde-abdb-4290-afbb-b1a8fc43a66c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:16.187523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:16.224427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ce508-8288-4f74-9c42-c80504426537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:16.225116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce508-8288-4f74-9c42-c80504426537 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.225294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce508-8288-4f74-9c42-c80504426537 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.225561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce508-8288-4f74-9c42-c80504426537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.314597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ce508-8288-4f74-9c42-c80504426537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:16.316092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ce508-8288-4f74-9c42-c80504426537 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:16.436331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 345/685] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:16.442014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:16.565704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3b00-8198-405a-b606-715de7adfc0b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:16.584612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:16.603115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:16.603749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.604029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.604300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.616481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:16.618053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:16.635931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:16.636955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:16.637900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:16.637900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:16.637900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:16.638539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 341/686] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:16.643205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ce508-8288-4f74-9c42-c80504426537 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:16.645360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:16.763903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ce508-8288-4f74-9c42-c80504426537 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:16.803852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:16.804774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.805146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.805546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.806853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:16.807253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:16.808715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:16.826373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 346/687] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:16.827953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.068203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936977-182c-43bb-82cc-e9c20665b51a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.120555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa223e-66b7-4b50-83fd-aa2174eb65f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.121202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa223e-66b7-4b50-83fd-aa2174eb65f1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.125627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa223e-66b7-4b50-83fd-aa2174eb65f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.125627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa223e-66b7-4b50-83fd-aa2174eb65f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.136606 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa223e-66b7-4b50-83fd-aa2174eb65f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.140173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa223e-66b7-4b50-83fd-aa2174eb65f1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-4faa223e-66b7-4b50-83fd-aa2174eb65f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:17.159842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:17.160803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:17.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 342/688] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:17.170756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.283892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.371035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c882e5-7462-47f8-a37c-0c42aa4a1b7a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.451471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.452436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.452930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.453429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.454653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:17.455092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.457666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.475540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55ebaa-1137-494c-a060-d69b19c539c5 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.476027 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55ebaa-1137-494c-a060-d69b19c539c5 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.476503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55ebaa-1137-494c-a060-d69b19c539c5 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.479036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 343/689] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:17.485368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bec9b3-a2d9-466f-b3c6-f40266ccc28c None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.485831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bec9b3-a2d9-466f-b3c6-f40266ccc28c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.486122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bec9b3-a2d9-466f-b3c6-f40266ccc28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.499959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 347/690] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:17.504627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.527266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-02bec9b3-a2d9-466f-b3c6-f40266ccc28c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:17.528769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-02bec9b3-a2d9-466f-b3c6-f40266ccc28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:17.596789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.725136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 344/691] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:17.727250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f577db1-f829-4064-90c5-ff840cd979d7 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.770830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785f390-ed7f-409a-98c4-6678162f90fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.771440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f390-ed7f-409a-98c4-6678162f90fb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.771720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f390-ed7f-409a-98c4-6678162f90fb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.772000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785f390-ed7f-409a-98c4-6678162f90fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.772728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785f390-ed7f-409a-98c4-6678162f90fb None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=7d93aa681e2a4427a61fe0bf472c60ec) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:17.773041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785f390-ed7f-409a-98c4-6678162f90fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.774671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785f390-ed7f-409a-98c4-6678162f90fb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.778003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.781747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 348/692] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/role_assignments?user.id=7d93aa681e2a4427a61fe0bf472c60ec&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:18.042403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723624-754c-4b3e-9201-48f7dab5f814 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.042743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723624-754c-4b3e-9201-48f7dab5f814 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.042968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73723624-754c-4b3e-9201-48f7dab5f814 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.044578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 345/693] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:18.050434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa0781-cd63-47ba-b2c6-d9184fda90af None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.050712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa0781-cd63-47ba-b2c6-d9184fda90af None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.050981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa0781-cd63-47ba-b2c6-d9184fda90af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.079660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-07aa0781-cd63-47ba-b2c6-d9184fda90af None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:18.080890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-07aa0781-cd63-47ba-b2c6-d9184fda90af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:18.093943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723624-754c-4b3e-9201-48f7dab5f814 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.177371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 349/694] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:18.224498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73723624-754c-4b3e-9201-48f7dab5f814 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.271997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.272568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.272847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.273119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.284295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.285770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:18.300929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:18.301928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:18.302993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:18.302993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:18.302993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:18.302993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 346/695] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:18.310163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa0781-cd63-47ba-b2c6-d9184fda90af None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.418429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa0781-cd63-47ba-b2c6-d9184fda90af None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.436376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.457817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.458423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.458701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.458969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.459681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:18.459984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.461342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.475285 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 350/696] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:18.481560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.497739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.534954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.659093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae3e462-06fe-4604-a3c0-2157a1d35a84 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.701200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5416e6-d813-4fc9-ad46-97f420368452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.702223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5416e6-d813-4fc9-ad46-97f420368452 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.702600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5416e6-d813-4fc9-ad46-97f420368452 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.703309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5416e6-d813-4fc9-ad46-97f420368452 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.704580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5416e6-d813-4fc9-ad46-97f420368452 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.706233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5416e6-d813-4fc9-ad46-97f420368452 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.706850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ec9f8-25f0-4ca4-9005-36104c7f7059 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.734317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 347/697] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/users => generated 254 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:18.736059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e03e-54dc-4a23-92d4-0d75eedcd6bf None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.736582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e03e-54dc-4a23-92d4-0d75eedcd6bf None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.736967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e03e-54dc-4a23-92d4-0d75eedcd6bf None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.739441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 348/698] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:18.746074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb039d21-bca8-46ca-b015-6e9ce1902561 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.746684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb039d21-bca8-46ca-b015-6e9ce1902561 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.747159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb039d21-bca8-46ca-b015-6e9ce1902561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.757657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbad23-358d-4787-b5df-c1792fc6c20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.757657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbad23-358d-4787-b5df-c1792fc6c20e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.757978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbad23-358d-4787-b5df-c1792fc6c20e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.758054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbad23-358d-4787-b5df-c1792fc6c20e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.772023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbad23-358d-4787-b5df-c1792fc6c20e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-7ecbad23-358d-4787-b5df-c1792fc6c20e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:18.778560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:18.779632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:18.780774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 351/699] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:18.794833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbad23-358d-4787-b5df-c1792fc6c20e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.799280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-cb039d21-bca8-46ca-b015-6e9ce1902561 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:18.800924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cb039d21-bca8-46ca-b015-6e9ce1902561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:18.836350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.918074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbad23-358d-4787-b5df-c1792fc6c20e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.941946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.959796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 349/700] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:18.959994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217bc7e-c9fb-4b6e-ad28-44dff1aad5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.960471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217bc7e-c9fb-4b6e-ad28-44dff1aad5f2 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.960751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217bc7e-c9fb-4b6e-ad28-44dff1aad5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.961026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217bc7e-c9fb-4b6e-ad28-44dff1aad5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.965841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217bc7e-c9fb-4b6e-ad28-44dff1aad5f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:18.966224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217bc7e-c9fb-4b6e-ad28-44dff1aad5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.967752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217bc7e-c9fb-4b6e-ad28-44dff1aad5f2 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.986362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 352/701] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:18.994831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb039d21-bca8-46ca-b015-6e9ce1902561 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.130285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb039d21-bca8-46ca-b015-6e9ce1902561 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.150847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47add80-daeb-4063-b941-aeeef837c089 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.151171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47add80-daeb-4063-b941-aeeef837c089 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.151398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47add80-daeb-4063-b941-aeeef837c089 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.153035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 353/702] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:19.158376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a635a0-c9ab-4b52-b8f6-652016e81dcf None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.158735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a635a0-c9ab-4b52-b8f6-652016e81dcf None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.159160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a635a0-c9ab-4b52-b8f6-652016e81dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.179659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a4a635a0-c9ab-4b52-b8f6-652016e81dcf None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:19.181441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a4a635a0-c9ab-4b52-b8f6-652016e81dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:19.181737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.182390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.182666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.182931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.199319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.201104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:19.222293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:19.223313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:19.224486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:19.224486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:19.224486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:19.224486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 350/703] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:19.232391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.273848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 354/704] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:19.279696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a635a0-c9ab-4b52-b8f6-652016e81dcf None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.305177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.319886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.365706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a2aba-1f47-49ee-84fe-4159361e5755 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.401549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.402445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.402723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.403000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.403653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:19.403962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.405343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.405849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a635a0-c9ab-4b52-b8f6-652016e81dcf None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.433213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 351/705] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:19.439889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.479648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.480291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.480565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.480838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] PATH_INFO: `/v3/services/network` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.481429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.484738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:19.509159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:19.510270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:19.511143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 355/706] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/services/network => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:19.516311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.614524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9e4f0-3e07-4818-a5c1-ec49351ac7e8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.626940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.654504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.655509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.656021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.656502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] PATH_INFO: `/v3/users/nova` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.660528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.667725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.669833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.670188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a755ed-7ec7-4375-861a-45e76b98bc5b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:19.684032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:19.685039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:19.686421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:19.688826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 352/707] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:19.690021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.708911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.709523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.709826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.710094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.711005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] RBAC: Adding query filter params (name=network) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:19.711312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.712802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.718104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 356/708] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:19.718913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.808200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94069a4-dc4d-475a-9eb1-3a111764b14b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.840460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dde6f0-d497-48ab-83d2-b5b02291e157 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.847026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.848014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.848556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.849079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.849915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.851654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:19.864984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:19.866166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:19.867039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:19.875418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 353/709] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:19.875418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.880669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.881592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.882169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.882644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.883639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:19.884258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.886419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.908704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 357/710] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:19.909997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.915931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.938199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:20.004033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a1e1d-a733-473f-9123-4a443776fc9c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:20.045726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d1191-8e37-4581-95fc-f89704451c59 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:20.049473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:20.050115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.050393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.050662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.051285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] RBAC: Adding query filter params (type=network) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:20.051586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:20.052961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:20.060852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 354/711] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:20.061795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:20.112024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:20.113073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.114239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.114737 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.115892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:20.116613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:20.120785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:20.132952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 358/712] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:20.139505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:20.283296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:20.294760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:20.300705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017d69f-947c-4ce2-8e18-187a9666583a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:20.315242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f88b520-c0ea-403a-8672-eeccb588d041 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:20.342315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d443ec0-b2c8-4b27-8afd-3d525e2a7748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:20.342820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443ec0-b2c8-4b27-8afd-3d525e2a7748 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.343104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443ec0-b2c8-4b27-8afd-3d525e2a7748 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.343374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d443ec0-b2c8-4b27-8afd-3d525e2a7748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.362051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:20.363801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.364341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.364976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.366052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:20.366675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:20.368927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:20.376821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 359/713] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 13 19:36:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:20.444142 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443ec0-b2c8-4b27-8afd-3d525e2a7748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:20.445925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d443ec0-b2c8-4b27-8afd-3d525e2a7748 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:20.606610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:20.613268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 355/714] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:20.618732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:20.755902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1fc29-a628-4627-8ce5-d1a73f36a88d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:20.817092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb98671d-16f5-4381-88a1-80b08b8deb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:20.818182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98671d-16f5-4381-88a1-80b08b8deb83 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.818693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98671d-16f5-4381-88a1-80b08b8deb83 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.819169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb98671d-16f5-4381-88a1-80b08b8deb83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.831403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98671d-16f5-4381-88a1-80b08b8deb83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:20.833945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb98671d-16f5-4381-88a1-80b08b8deb83 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:20.838701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-fb98671d-16f5-4381-88a1-80b08b8deb83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:20.856503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:20.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:20.858551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:20.858551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:20.858551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:20.859159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 360/715] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:20.864686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d443ec0-b2c8-4b27-8afd-3d525e2a7748 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:20.927653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9ddb-d9fc-412d-b020-74f63aed0dc8 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.928155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9ddb-d9fc-412d-b020-74f63aed0dc8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.928649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c9ddb-d9fc-412d-b020-74f63aed0dc8 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.933766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 361/716] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:20.938316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38016882-f334-4af8-9eaa-6cbfaf6cbb76 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.938822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38016882-f334-4af8-9eaa-6cbfaf6cbb76 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.939324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38016882-f334-4af8-9eaa-6cbfaf6cbb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.982737 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-38016882-f334-4af8-9eaa-6cbfaf6cbb76 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:20.983959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-38016882-f334-4af8-9eaa-6cbfaf6cbb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:21.067067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d443ec0-b2c8-4b27-8afd-3d525e2a7748 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.112015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 362/717] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:21.124889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.125515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.125831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.126105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.126764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:21.127072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.128501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.141671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 356/718] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:21.150889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38016882-f334-4af8-9eaa-6cbfaf6cbb76 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.264429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38016882-f334-4af8-9eaa-6cbfaf6cbb76 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.294028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.303772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.304410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.304692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.304962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.312327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.313795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:21.326088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:21.326995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:21.327927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 363/719] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:21.334914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.415049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.467278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97916589-9a75-4fa8-85cf-6f82845e4a7e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.506340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9bf92-0711-489d-a259-2540b21aa15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.506957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9bf92-0711-489d-a259-2540b21aa15c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.507238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9bf92-0711-489d-a259-2540b21aa15c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.507508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9bf92-0711-489d-a259-2540b21aa15c None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.508199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9bf92-0711-489d-a259-2540b21aa15c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:21.509105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9bf92-0711-489d-a259-2540b21aa15c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.510710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9bf92-0711-489d-a259-2540b21aa15c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.530900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 357/720] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:21.536589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.643158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de15a58-d891-4f7d-bc05-2995b72e51e6 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.643664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de15a58-d891-4f7d-bc05-2995b72e51e6 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.644166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de15a58-d891-4f7d-bc05-2995b72e51e6 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.647078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 358/721] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:21.648467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b6cc3-7380-49f1-83b3-a091469acd75 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.656457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5510d38-dd73-43e9-964a-68c59972e08a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.656946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5510d38-dd73-43e9-964a-68c59972e08a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.657428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5510d38-dd73-43e9-964a-68c59972e08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.684542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.685071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.685347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.685634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.686359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=2981bb0558d34329a061a5f08149bba3) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:21.686666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.688100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.688520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e5510d38-dd73-43e9-964a-68c59972e08a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:21.690594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e5510d38-dd73-43e9-964a-68c59972e08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:21.694173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 364/722] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/role_assignments?user.id=2981bb0558d34329a061a5f08149bba3&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=b698800ceec3460a8519ea627650da0f => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:21.736013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.777265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 359/723] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:21.796356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.856666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7b0ba-5cd2-4ca7-93fe-c83f13fb1ac0 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.897328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa42d8-c322-4524-93c0-692af453c1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.898071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa42d8-c322-4524-93c0-692af453c1ba None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.898352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa42d8-c322-4524-93c0-692af453c1ba None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.898623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa42d8-c322-4524-93c0-692af453c1ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.910610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aa42d8-c322-4524-93c0-692af453c1ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-11aa42d8-c322-4524-93c0-692af453c1ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:21.917668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:21.918620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:21.919498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 365/724] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:21.924900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5510d38-dd73-43e9-964a-68c59972e08a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.989867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.063873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5510d38-dd73-43e9-964a-68c59972e08a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.105743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.106369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.106649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.106922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.107648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:22.107958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.109405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.123935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 360/725] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:22.129882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa42d8-c322-4524-93c0-692af453c1ba None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.262995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.281901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa42d8-c322-4524-93c0-692af453c1ba None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.298935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.352273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.353759 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.354035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.354299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.379593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.382564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:22.411838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:22.412863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:22.413828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:22.413828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:22.413828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:22.413828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 366/726] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:22.419084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.462235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b906990-2013-41d2-976b-8f19bde09cac None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.484134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.533495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb02eb-289c-4152-aeef-b46b07b1f3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.534118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb02eb-289c-4152-aeef-b46b07b1f3b4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.534396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb02eb-289c-4152-aeef-b46b07b1f3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.534666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb02eb-289c-4152-aeef-b46b07b1f3b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.555274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb02eb-289c-4152-aeef-b46b07b1f3b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-f3eb02eb-289c-4152-aeef-b46b07b1f3b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:22.565409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:22.568152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:22.570722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 361/727] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:22.577634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb02eb-289c-4152-aeef-b46b07b1f3b4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.638943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de4af4-1e35-4fa0-a62d-60e03e13f102 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.715887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.716509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.716784 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.717059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.717742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:22.718042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.719409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.752843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 367/728] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:22.760960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.763730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb02eb-289c-4152-aeef-b46b07b1f3b4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.802967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.835579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.836264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.836601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.837464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.838165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:22.838471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.839886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.849399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 362/729] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:22.854852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.890270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.962345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc3633-7e93-43bc-8c17-6f4b3a615572 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.969482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2b594-acb6-4c1b-ba8c-632d7654042c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.999041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c62a3-e700-4d7c-bd9f-6128c8395373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.999812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c62a3-e700-4d7c-bd9f-6128c8395373 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.000050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c62a3-e700-4d7c-bd9f-6128c8395373 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.000325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c62a3-e700-4d7c-bd9f-6128c8395373 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.009572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c62a3-e700-4d7c-bd9f-6128c8395373 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.010787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e4c9b-e320-4f34-9284-cafbb3493d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.011062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c62a3-e700-4d7c-bd9f-6128c8395373 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.011933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4c9b-e320-4f34-9284-cafbb3493d0a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.012442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4c9b-e320-4f34-9284-cafbb3493d0a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.013034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4c9b-e320-4f34-9284-cafbb3493d0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.025132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e4c9b-e320-4f34-9284-cafbb3493d0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.026740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e4c9b-e320-4f34-9284-cafbb3493d0a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-8e1c62a3-e700-4d7c-bd9f-6128c8395373 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:23.034983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:23.036056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:23.037061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:23.041501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 368/730] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:23.042696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768346-bea0-4652-8e27-2263a9443c77 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.042978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768346-bea0-4652-8e27-2263a9443c77 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.043251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768346-bea0-4652-8e27-2263a9443c77 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-eb3e4c9b-e320-4f34-9284-cafbb3493d0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:23.043910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:23.045055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:23.046469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:23.046469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:23.046469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:23.046644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 369/731] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:23.048525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11768346-bea0-4652-8e27-2263a9443c77 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.048690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 363/732] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:23.053505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8ddfe-c0f1-4bf2-9846-17eecd2a6b5d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.053755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8ddfe-c0f1-4bf2-9846-17eecd2a6b5d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.054013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8ddfe-c0f1-4bf2-9846-17eecd2a6b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.081546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-97c8ddfe-c0f1-4bf2-9846-17eecd2a6b5d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:23.083695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-97c8ddfe-c0f1-4bf2-9846-17eecd2a6b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:23.144927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.160872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 364/733] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:23.162131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8ddfe-c0f1-4bf2-9846-17eecd2a6b5d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.177121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11768346-bea0-4652-8e27-2263a9443c77 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.179781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.258569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.259564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.260021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.260470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.261544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:23.262049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.263802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.287517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c8ddfe-c0f1-4bf2-9846-17eecd2a6b5d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.302443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 370/734] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:23.309830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.336151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff176a8-5ed3-4ea9-b54b-c57511a275db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.341856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff176a8-5ed3-4ea9-b54b-c57511a275db None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.342171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff176a8-5ed3-4ea9-b54b-c57511a275db None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.342458 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff176a8-5ed3-4ea9-b54b-c57511a275db None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.343134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff176a8-5ed3-4ea9-b54b-c57511a275db None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:23.343445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff176a8-5ed3-4ea9-b54b-c57511a275db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.344852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff176a8-5ed3-4ea9-b54b-c57511a275db None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.365689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 365/735] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:23.367526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9529ab-bb90-490f-bcfc-60ce608a9bb7 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.368601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9529ab-bb90-490f-bcfc-60ce608a9bb7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.369848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9529ab-bb90-490f-bcfc-60ce608a9bb7 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.375294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 366/736] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:23.376315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9529ab-bb90-490f-bcfc-60ce608a9bb7 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.454394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c4c4d-e26d-4739-a87b-e977469a8771 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.490572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.496792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436024d-bffd-4d5e-9b1d-11edb20f466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.497425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436024d-bffd-4d5e-9b1d-11edb20f466e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.497764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436024d-bffd-4d5e-9b1d-11edb20f466e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.498360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436024d-bffd-4d5e-9b1d-11edb20f466e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.516779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.541255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9529ab-bb90-490f-bcfc-60ce608a9bb7 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.583075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b84d37-77d5-423f-b3d2-f85401034db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.583639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b84d37-77d5-423f-b3d2-f85401034db0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.583920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b84d37-77d5-423f-b3d2-f85401034db0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.584190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b84d37-77d5-423f-b3d2-f85401034db0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.592931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b84d37-77d5-423f-b3d2-f85401034db0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.594436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b84d37-77d5-423f-b3d2-f85401034db0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.596259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436024d-bffd-4d5e-9b1d-11edb20f466e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.598886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436024d-bffd-4d5e-9b1d-11edb20f466e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-69b84d37-77d5-423f-b3d2-f85401034db0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:23.607224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:23.608315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:23.609279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:23.613075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 367/737] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:23.614395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfed3ae-a5ff-469f-a28b-9c86ea06119a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.614688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfed3ae-a5ff-469f-a28b-9c86ea06119a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.614970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfed3ae-a5ff-469f-a28b-9c86ea06119a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.634094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ebfed3ae-a5ff-469f-a28b-9c86ea06119a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:23.635413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ebfed3ae-a5ff-469f-a28b-9c86ea06119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:23.726014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 368/738] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:23.726957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfed3ae-a5ff-469f-a28b-9c86ea06119a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.731904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 371/739] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:23.732754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436024d-bffd-4d5e-9b1d-11edb20f466e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.782673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.806991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.849143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfed3ae-a5ff-469f-a28b-9c86ea06119a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.888084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854dca1-016c-4b93-bbbe-21fefa496552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.889143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854dca1-016c-4b93-bbbe-21fefa496552 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.889718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854dca1-016c-4b93-bbbe-21fefa496552 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.890004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436024d-bffd-4d5e-9b1d-11edb20f466e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.890449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854dca1-016c-4b93-bbbe-21fefa496552 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.891323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854dca1-016c-4b93-bbbe-21fefa496552 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.892943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854dca1-016c-4b93-bbbe-21fefa496552 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.922275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 369/740] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:23 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:23.922275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854dca1-016c-4b93-bbbe-21fefa496552 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.973889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.974591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.974876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.975156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.975898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:23.976206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.978508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.020738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 372/741] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:24.021599 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.079176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854dca1-016c-4b93-bbbe-21fefa496552 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.117375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.138733 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.139386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.139693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.139966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.153462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.154700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.155373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.170183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c1d3a-da3e-441e-8b88-8521691b89e6 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:24.172194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:24.173117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:24.179335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:24.179335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:24.179335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:24.179335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 370/742] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:24.179335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.216835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcba44-9ee8-4639-82c1-99f4e8122890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.217451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcba44-9ee8-4639-82c1-99f4e8122890 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.217765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcba44-9ee8-4639-82c1-99f4e8122890 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.218040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dcba44-9ee8-4639-82c1-99f4e8122890 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.218704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcba44-9ee8-4639-82c1-99f4e8122890 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.220109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dcba44-9ee8-4639-82c1-99f4e8122890 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.237653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 373/743] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:24 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:24.238483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcba44-9ee8-4639-82c1-99f4e8122890 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.314518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79de15-9796-4ad4-be77-ff133e6c42f5 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.373687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ae3b6-7096-431a-bd63-0a6aef293309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.375145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ae3b6-7096-431a-bd63-0a6aef293309 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.375668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ae3b6-7096-431a-bd63-0a6aef293309 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.376140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ae3b6-7096-431a-bd63-0a6aef293309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.400404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dcba44-9ee8-4639-82c1-99f4e8122890 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.431047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.440180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7fa1e6-8dac-4421-b89c-bf241b78e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.440811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fa1e6-8dac-4421-b89c-bf241b78e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.441090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fa1e6-8dac-4421-b89c-bf241b78e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.441362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7fa1e6-8dac-4421-b89c-bf241b78e3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.442153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7fa1e6-8dac-4421-b89c-bf241b78e3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:24.442466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7fa1e6-8dac-4421-b89c-bf241b78e3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.443885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7fa1e6-8dac-4421-b89c-bf241b78e3f5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.457283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 374/744] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:24.458645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f27a5-eac2-41a6-a433-65bac2fde28a None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.458887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f27a5-eac2-41a6-a433-65bac2fde28a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.459160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f27a5-eac2-41a6-a433-65bac2fde28a None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.460387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 375/745] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:24.467477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f27a5-eac2-41a6-a433-65bac2fde28a None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.469776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ae3b6-7096-431a-bd63-0a6aef293309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.471531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ae3b6-7096-431a-bd63-0a6aef293309 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.484084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.592086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 371/746] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:36:24.593766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329f7db-ed9f-40c4-8bb6-86eb304964de None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.594371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329f7db-ed9f-40c4-8bb6-86eb304964de None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.594840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329f7db-ed9f-40c4-8bb6-86eb304964de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.609974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f27a5-eac2-41a6-a433-65bac2fde28a None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.614756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2329f7db-ed9f-40c4-8bb6-86eb304964de None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:24.616183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2329f7db-ed9f-40c4-8bb6-86eb304964de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:24.653807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.654367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.654644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.654919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.663842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.666679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:24.681954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:24.682917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 376/747] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:24.684574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.709703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 372/748] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:24.711082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329f7db-ed9f-40c4-8bb6-86eb304964de None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.717506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.831428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.891570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbbe26f-b66b-437c-86a5-714ef35789a0 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.933931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329f7db-ed9f-40c4-8bb6-86eb304964de None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.969622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.970464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.970940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.972971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.021443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.023226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.044354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.044989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.045268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.045541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.046271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:25.046575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:25.053244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:25.054258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:25.055191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:25.055191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:25.055191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:25.055374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.055446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 377/749] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:25.062848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.079350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 373/750] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:25.090264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.207339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.220264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4915e-c318-4cb7-a84b-ea5132e87fa9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.251036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.264304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb754fa9-dbfc-4136-82b4-e3ed24be90a3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.268004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.268830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.269260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.269791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.270590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:25.271067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.273263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.295510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 378/751] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:25.295510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.314846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.316041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.316584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.317068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.318098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=7d93aa681e2a4427a61fe0bf472c60ec) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:25.318636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.321400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.329192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 374/752] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/role_assignments?user.id=7d93aa681e2a4427a61fe0bf472c60ec&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=b698800ceec3460a8519ea627650da0f => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:25.330681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.411886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43de60c-1201-4e2d-a1a6-69ec8ed1e641 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.451608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e04d4-3cd3-4072-bb37-565fb5743a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.452254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e04d4-3cd3-4072-bb37-565fb5743a4d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.452539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e04d4-3cd3-4072-bb37-565fb5743a4d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.452817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764e04d4-3cd3-4072-bb37-565fb5743a4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.465921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764e04d4-3cd3-4072-bb37-565fb5743a4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-764e04d4-3cd3-4072-bb37-565fb5743a4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:25.471799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:25.472796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:25.474138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 379/753] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:25.487760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3199872-2a01-4a1a-bde6-07a0dc94b68e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.491911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e04d4-3cd3-4072-bb37-565fb5743a4d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.536558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.555855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.558253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.559620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.560188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.568471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.570807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.582798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:25.583750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:25.584615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:25.585522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:25.585522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:25.585522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:25.585522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:25.585522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:25.585522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:25.585522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:25.585522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:25.593205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 375/754] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:25.594746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.671743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764e04d4-3cd3-4072-bb37-565fb5743a4d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.744489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd6f62-f8f5-45fc-aede-d35dd25e7eeb None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.748880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.749511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.749816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.750089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.750766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:25.751067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.752443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.776129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 380/755] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:25.781218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.792981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.794085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.794572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.795044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.796041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:25.797022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.798720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.817998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 376/756] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:25.821481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.878108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.970024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2d304-dfa8-4010-bb16-2094fccb17de None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.986371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e1aab-e9a3-4df4-80ef-e9869764c90a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.995536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.036763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.037571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.037885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.038152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.053082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.054089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.054582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.056495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.057470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=803b35616c55468eb7a0f72c9c86b72b) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:26.058016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.059923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.064880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.066668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.072807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 377/757] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/role_assignments?user.id=803b35616c55468eb7a0f72c9c86b72b&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:26.095207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:26.096270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:26.097198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:26.097198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:26.097198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:26.099240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 381/758] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:26.106057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.257704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ae0ec-0869-464c-b889-2ebb1f2d3f9c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.276574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.306032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cef6cc-5698-4ece-939a-4fd70d0f6dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.306729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cef6cc-5698-4ece-939a-4fd70d0f6dac None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.307019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cef6cc-5698-4ece-939a-4fd70d0f6dac None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.307296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cef6cc-5698-4ece-939a-4fd70d0f6dac None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.307973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cef6cc-5698-4ece-939a-4fd70d0f6dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:26.308284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cef6cc-5698-4ece-939a-4fd70d0f6dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.310671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cef6cc-5698-4ece-939a-4fd70d0f6dac None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.331849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 378/759] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:26.333250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.448162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe55907-2f52-4867-bc35-7a3e93574267 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.477814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2145c-108e-4078-947a-68394422d079 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.478175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2145c-108e-4078-947a-68394422d079 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.478365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2145c-108e-4078-947a-68394422d079 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.479715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 379/760] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:26.485505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf8f34-7d93-4888-9fc8-a7fdc0b3551b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.485839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf8f34-7d93-4888-9fc8-a7fdc0b3551b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.486123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbf8f34-7d93-4888-9fc8-a7fdc0b3551b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.503357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.504010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.504287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.504556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.513176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.514474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7fbf8f34-7d93-4888-9fc8-a7fdc0b3551b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:26.514592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.516120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7fbf8f34-7d93-4888-9fc8-a7fdc0b3551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:26.541983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:26.543129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:26.544513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:26.551726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 382/761] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:26.558293 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.603081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.683122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 380/762] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:26.730480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24325d67-1514-45aa-8140-55bdc65826b4 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.770585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e89e8c-43b7-409a-ba0b-e7ff1d2b3e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.771572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e89e8c-43b7-409a-ba0b-e7ff1d2b3e03 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.772064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e89e8c-43b7-409a-ba0b-e7ff1d2b3e03 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.772538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e89e8c-43b7-409a-ba0b-e7ff1d2b3e03 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.773536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e89e8c-43b7-409a-ba0b-e7ff1d2b3e03 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:26.774138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e89e8c-43b7-409a-ba0b-e7ff1d2b3e03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.775789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e89e8c-43b7-409a-ba0b-e7ff1d2b3e03 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.800150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 383/763] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:26.805728 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf8f34-7d93-4888-9fc8-a7fdc0b3551b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.944893 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae53b96-6d53-4d44-900a-33ce7014a55d None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.945852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae53b96-6d53-4d44-900a-33ce7014a55d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.946372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae53b96-6d53-4d44-900a-33ce7014a55d None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.947932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 384/764] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:26.953581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821a5ff-e07d-44ff-8724-0aa57b521697 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.953882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821a5ff-e07d-44ff-8724-0aa57b521697 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.954154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821a5ff-e07d-44ff-8724-0aa57b521697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.973839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbf8f34-7d93-4888-9fc8-a7fdc0b3551b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.977964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.992173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-8821a5ff-e07d-44ff-8724-0aa57b521697 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:26.994718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8821a5ff-e07d-44ff-8724-0aa57b521697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:27.055739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.056758 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.057247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.057743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.080637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.082531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.109357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 385/765] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:27.110818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:27.112093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:27.112993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:27.112993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:27.112993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:27.115033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 381/766] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:27.123721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821a5ff-e07d-44ff-8724-0aa57b521697 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.134551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.140550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.299715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821a5ff-e07d-44ff-8724-0aa57b521697 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.340209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.341206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.341724 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.342386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.344474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:27.344991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.347348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.365313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 386/767] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:27.366357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32525773-c59f-4d85-84a1-e404c43ae6f5 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.374530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.427662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.428321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.428603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.428883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] PATH_INFO: `/v3/services/network` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.429668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.431398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:27.449168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:27.450251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:27.451190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:27.455807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 382/768] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/services/network => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:27.456671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.459952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.481658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.503016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737a79e-a162-421d-86f1-88658fc10baa None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.547375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.548142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.548543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.548952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.556886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.558839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:27.580488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:27.581493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:27.582489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:27.582489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:27.582489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:27.582489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:27.582489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:27.582489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:27.582489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:27.582489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:27.586466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 387/769] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:27.587475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.603108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2090f9-fb9c-482e-bbaa-d12d9a088fb8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.660323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.660959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.661240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.661510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.662121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.663499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:27.688368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:27.690207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:27.691273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:27.693711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 383/770] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:27.694896 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.727509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64324a-2f22-4c6b-84cb-61878cd7064d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.755338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.768871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.769911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.770350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.770836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.771863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] RBAC: Adding query filter params (name=network) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:27.772378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.774926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.789777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 388/771] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:27.791496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.810618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.837915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5d05-ce0e-4471-8899-2459db9fd60d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.886796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.888048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.889468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.889946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.890832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:27.891345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.892966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.911423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 384/772] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:27.912600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.927555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbf567-0ce5-4b6a-ac69-38bc884ca860 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.966601 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.967381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.967777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.968120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.968877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:27.969247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.970793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.981675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 389/773] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:27.982868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.054210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.061282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfe74f5-9bbf-49cc-b291-1d7dcc013949 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.109813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.110589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6777bd-37b6-4c5a-9fef-98fc1c41e997 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.110944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.111777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.112510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.114054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] RBAC: Adding query filter params (type=network) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:28.114210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.115612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.123771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 385/774] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:28.124496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.130691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.152808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04c6612-86ea-4f9b-a896-7da355900495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.153403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c6612-86ea-4f9b-a896-7da355900495 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.153704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c6612-86ea-4f9b-a896-7da355900495 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.153986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04c6612-86ea-4f9b-a896-7da355900495 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/2981bb0558d34329a061a5f08149bba3/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.155889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c6612-86ea-4f9b-a896-7da355900495 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=2981bb0558d34329a061a5f08149bba3, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.157320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04c6612-86ea-4f9b-a896-7da355900495 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.178144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 390/775] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:27 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/2981bb0558d34329a061a5f08149bba3/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:28.178144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d056-056c-49aa-a610-77abadfa72a8 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.178144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d056-056c-49aa-a610-77abadfa72a8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.178144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d056-056c-49aa-a610-77abadfa72a8 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.178144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 391/776] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:28.178604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179d056-056c-49aa-a610-77abadfa72a8 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.249860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6f08b-a2bd-4bbf-9b06-25e48b31eb64 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.297239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858b3d0-c2eb-40bc-8750-a4e03bc132e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.297895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b3d0-c2eb-40bc-8750-a4e03bc132e4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.298177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b3d0-c2eb-40bc-8750-a4e03bc132e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.298446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b3d0-c2eb-40bc-8750-a4e03bc132e4 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.299199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b3d0-c2eb-40bc-8750-a4e03bc132e4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:28.299506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b3d0-c2eb-40bc-8750-a4e03bc132e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.300919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b3d0-c2eb-40bc-8750-a4e03bc132e4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.307439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 386/777] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:28.309030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07bc77-f047-4fa6-82b6-e1a55ca73882 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.309319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07bc77-f047-4fa6-82b6-e1a55ca73882 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.309588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07bc77-f047-4fa6-82b6-e1a55ca73882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.327636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179d056-056c-49aa-a610-77abadfa72a8 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.331085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-4d07bc77-f047-4fa6-82b6-e1a55ca73882 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:28.332369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4d07bc77-f047-4fa6-82b6-e1a55ca73882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:28.346440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.375933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.376824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.377170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.377512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.378332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] RBAC: Adding query filter params (interface=public, service_id=3b781e60d9f74ebabd0830e652c2723d, region_id=RegionOne) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:28.378707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.382281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.392003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 392/778] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/endpoints?service_id=3b781e60d9f74ebabd0830e652c2723d&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:28.392931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.416085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.422443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 387/779] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:28.423400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07bc77-f047-4fa6-82b6-e1a55ca73882 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.517659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcc325-2369-4799-ad65-a5586e45df41 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.530134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07bc77-f047-4fa6-82b6-e1a55ca73882 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.562458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41733985-96fd-496b-8715-803abf87c708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.562458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41733985-96fd-496b-8715-803abf87c708 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.562458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41733985-96fd-496b-8715-803abf87c708 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.562458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41733985-96fd-496b-8715-803abf87c708 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.562898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41733985-96fd-496b-8715-803abf87c708 None admin] RBAC: Adding query filter params (interface=public, service_id=54a444f73e144f25a1c92aa75192b17a, region_id=RegionOne) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:28.562898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41733985-96fd-496b-8715-803abf87c708 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.564176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41733985-96fd-496b-8715-803abf87c708 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.571492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 393/780] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/endpoints?service_id=54a444f73e144f25a1c92aa75192b17a&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:28.583763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41733985-96fd-496b-8715-803abf87c708 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.585184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de007d1-ffc4-40eb-9fc6-0a6e88406ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.586093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de007d1-ffc4-40eb-9fc6-0a6e88406ab5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.587342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de007d1-ffc4-40eb-9fc6-0a6e88406ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.587924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de007d1-ffc4-40eb-9fc6-0a6e88406ab5 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.588737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de007d1-ffc4-40eb-9fc6-0a6e88406ab5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.590480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de007d1-ffc4-40eb-9fc6-0a6e88406ab5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.604409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 388/781] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:28.642118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.716459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad434-ce91-4140-af9f-1a5c9e12d209 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.716766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad434-ce91-4140-af9f-1a5c9e12d209 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.717017 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad434-ce91-4140-af9f-1a5c9e12d209 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.718742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 389/782] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:28.728588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7bac7-23d4-4142-9a5c-cd2d4dd6a10d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.728588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7bac7-23d4-4142-9a5c-cd2d4dd6a10d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.728588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7bac7-23d4-4142-9a5c-cd2d4dd6a10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.738559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41733985-96fd-496b-8715-803abf87c708 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.762695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-97b7bac7-23d4-4142-9a5c-cd2d4dd6a10d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:28.763970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-97b7bac7-23d4-4142-9a5c-cd2d4dd6a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:28.779783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a9f35-04b9-446b-b1db-3cddd5bfd957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.781336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a9f35-04b9-446b-b1db-3cddd5bfd957 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.781880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a9f35-04b9-446b-b1db-3cddd5bfd957 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.782362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a9f35-04b9-446b-b1db-3cddd5bfd957 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.783250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a9f35-04b9-446b-b1db-3cddd5bfd957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.784904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a9f35-04b9-446b-b1db-3cddd5bfd957 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.792308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 394/783] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:28.809261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a9f35-04b9-446b-b1db-3cddd5bfd957 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.878684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.894113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 390/784] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:28.977758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a9f35-04b9-446b-b1db-3cddd5bfd957 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.986070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.027930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4819a-a991-4f22-b333-452a651971c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.028595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4819a-a991-4f22-b333-452a651971c3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.028878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4819a-a991-4f22-b333-452a651971c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.029163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4819a-a991-4f22-b333-452a651971c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.040742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4819a-a991-4f22-b333-452a651971c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-9be4819a-a991-4f22-b333-452a651971c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:29.045845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:29.046818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:29.047681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 395/785] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:29.053208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7bac7-23d4-4142-9a5c-cd2d4dd6a10d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.201563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7bac7-23d4-4142-9a5c-cd2d4dd6a10d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.264849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.265405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.265711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.265986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.266674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:29.266980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.268715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.279789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 391/786] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:29.288112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4819a-a991-4f22-b333-452a651971c3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.301262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.401421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.489032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4819a-a991-4f22-b333-452a651971c3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.527933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea0e00-c63d-4368-a44a-a0b92f960b46 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.560220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.561234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.561773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.562276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.584628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.588139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.604989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865af58-152e-4f2d-af93-2ccd109ed486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.605657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865af58-152e-4f2d-af93-2ccd109ed486 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.605945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865af58-152e-4f2d-af93-2ccd109ed486 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.606221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865af58-152e-4f2d-af93-2ccd109ed486 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:29.619825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:29.620828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:29.621768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:29.621768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:29.621768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:29.622358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 396/787] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:29.635088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.644952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865af58-152e-4f2d-af93-2ccd109ed486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-a865af58-152e-4f2d-af93-2ccd109ed486 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:29.651410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:29.652547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:29.654714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:29.661679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.661966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 392/788] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:29.661966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865af58-152e-4f2d-af93-2ccd109ed486 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.761228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7088f09-03b7-4d03-ab85-c34b1e5a7b5b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.836170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.836824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.837279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.837556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.838256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:29.838563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.839992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.854385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865af58-152e-4f2d-af93-2ccd109ed486 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.874734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 397/789] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:29.880688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.886819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.931339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.931968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.932249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.932519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.933199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:29.933503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.934908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.953117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 393/790] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:29.958396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.003983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.049028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eab0b4-f339-4320-8bb4-a788fe583221 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.122201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.122844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.123118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.123387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.134370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.135815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:30.157659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:30.158673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:30.159622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:30.160640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 398/791] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:30.168789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b4fbb-a87e-4cca-86d7-6969b9f5d33c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.170600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.235046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.235746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.236041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.236319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.260197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.261701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:30.291067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:30.292068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:30.292972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:30.292972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:30.292972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:30.292972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 394/792] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:30.300083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.333019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c96fd-d4ea-428c-ba97-280941f1ee31 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.368858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.399200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.399781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.400073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.400343 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.401084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:30.401388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.402861 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.413071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.432192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 399/793] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.433980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.533958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fc103d-5969-4958-badc-6081b6113548 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.594558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87790c57-0bea-4e2e-ad08-6757207d903c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.604847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.605835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.606314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.606589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.607243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:30.607546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.608993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.637023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.638143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.638994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.639495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.641171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 395/794] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.646903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.656545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.659844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:30.690065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:30.691110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:30.692023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:30.692023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:30.692023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:30.704162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 400/795] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:30.708310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.803191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.810041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.827876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d25bc9-b9aa-4c55-8e5f-99fb84332060 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.873894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f8c81-1f35-49c6-9450-6986a75a6078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.874961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8c81-1f35-49c6-9450-6986a75a6078 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.875447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8c81-1f35-49c6-9450-6986a75a6078 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.875942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f8c81-1f35-49c6-9450-6986a75a6078 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.877056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee73b4-e195-4065-92ae-95ea48aaaa68 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.884905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f8c81-1f35-49c6-9450-6986a75a6078 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.886838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f8c81-1f35-49c6-9450-6986a75a6078 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-3d0f8c81-1f35-49c6-9450-6986a75a6078 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:30.900636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:30.902034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:30.902997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:30.905578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 396/796] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:30.907209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1971a-6238-4b45-8dbb-fac0c4127d04 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.907701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1971a-6238-4b45-8dbb-fac0c4127d04 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.908179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1971a-6238-4b45-8dbb-fac0c4127d04 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.910092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 397/797] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:30.911206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1971a-6238-4b45-8dbb-fac0c4127d04 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.922932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064f176-0737-4827-8e43-52785708a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.923547 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f176-0737-4827-8e43-52785708a27f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.923897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f176-0737-4827-8e43-52785708a27f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.924172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064f176-0737-4827-8e43-52785708a27f None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.925164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064f176-0737-4827-8e43-52785708a27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:30.925164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064f176-0737-4827-8e43-52785708a27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.926506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064f176-0737-4827-8e43-52785708a27f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.944138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 401/798] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.944138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc76061-4f2a-4094-99dd-8f40dbe6800c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.944138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc76061-4f2a-4094-99dd-8f40dbe6800c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.944138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc76061-4f2a-4094-99dd-8f40dbe6800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.962978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4bc76061-4f2a-4094-99dd-8f40dbe6800c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:30.963803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4bc76061-4f2a-4094-99dd-8f40dbe6800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:31.043132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 402/799] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:31.044014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc76061-4f2a-4094-99dd-8f40dbe6800c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.048450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1971a-6238-4b45-8dbb-fac0c4127d04 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.086426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.088705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.090532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.091031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.091504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.092457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:31.092974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.094699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.116303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 398/800] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.122935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.136508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.255653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04ba9d-e951-4f7d-9b8d-7aebf3fa4b02 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.263916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc76061-4f2a-4094-99dd-8f40dbe6800c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.300675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.301826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.302313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.302786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.314814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.316796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:31.334117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:31.335115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:31.336069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:31.336069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:31.336069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:31.338333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 399/801] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:31.343312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.357910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.359278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.359759 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.360228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.373376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.375529 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:31.392452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:31.393386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:31.394313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:31.394313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:31.394313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:31.394313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:31.394313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:31.394313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:31.394313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:31.394313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:31.403051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 403/802] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:31.404753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.434543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.482438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358fe82-e99b-4ed8-9acc-af886929aa19 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.523602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.524144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.524426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.524701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.525351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:31.525678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.527133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.531263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.534880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6867c5-849c-4967-b468-b91fb21c2a9b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.545989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 400/803] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.545989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.575087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313498cb-31ab-4370-b144-44a3fbd32856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.575720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313498cb-31ab-4370-b144-44a3fbd32856 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.575998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313498cb-31ab-4370-b144-44a3fbd32856 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.576270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313498cb-31ab-4370-b144-44a3fbd32856 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.586739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313498cb-31ab-4370-b144-44a3fbd32856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-313498cb-31ab-4370-b144-44a3fbd32856 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:31.592162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:31.593102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:31.593997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 404/804] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:31.595190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a756-e989-4140-b4d3-d0b5a57f77f6 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.595190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a756-e989-4140-b4d3-d0b5a57f77f6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.595190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a756-e989-4140-b4d3-d0b5a57f77f6 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.596335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 405/805] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:31.596852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64a756-e989-4140-b4d3-d0b5a57f77f6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.655211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bdb27-e118-4803-b4db-e64fcf0355d4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.673235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.690010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995f91c-6f32-467c-ac2e-fadc0a7fa926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.690618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f91c-6f32-467c-ac2e-fadc0a7fa926 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.690905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f91c-6f32-467c-ac2e-fadc0a7fa926 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.691182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995f91c-6f32-467c-ac2e-fadc0a7fa926 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.691895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995f91c-6f32-467c-ac2e-fadc0a7fa926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:31.692202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995f91c-6f32-467c-ac2e-fadc0a7fa926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.693658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995f91c-6f32-467c-ac2e-fadc0a7fa926 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.698420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64a756-e989-4140-b4d3-d0b5a57f77f6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.705214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 401/806] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.706581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d0f5-4f44-46d6-aec6-3c979264e35d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.706870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d0f5-4f44-46d6-aec6-3c979264e35d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.707148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d0f5-4f44-46d6-aec6-3c979264e35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.724784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e278d0f5-4f44-46d6-aec6-3c979264e35d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:31.726022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e278d0f5-4f44-46d6-aec6-3c979264e35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:31.731891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.732527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.732786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.733055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.739841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.741258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:31.752733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:31.753814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 406/807] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:31.754732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.775645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.795629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 402/808] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:31.796891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b36ad-08fe-44f7-b932-bab2a0e7a4d2 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.797175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b36ad-08fe-44f7-b932-bab2a0e7a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.797449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b36ad-08fe-44f7-b932-bab2a0e7a4d2 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.798701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 403/809] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:31.799379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b36ad-08fe-44f7-b932-bab2a0e7a4d2 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.871221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adbb64e-0d05-4521-bd6f-98dc0109d958 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.908861 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.909487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.909804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.910078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.910755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:31.911058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.912461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.927422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 407/810] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.927422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.021341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.034427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109fe9c-5d9a-4628-a30d-b41bd437bc3f None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.038535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b36ad-08fe-44f7-b932-bab2a0e7a4d2 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.048732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.069544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d30d3-4f2f-4c9d-b143-51c86a34fdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.070958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d30d3-4f2f-4c9d-b143-51c86a34fdea None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.071233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d30d3-4f2f-4c9d-b143-51c86a34fdea None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.071513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d30d3-4f2f-4c9d-b143-51c86a34fdea None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.072186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d30d3-4f2f-4c9d-b143-51c86a34fdea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:32.072483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d30d3-4f2f-4c9d-b143-51c86a34fdea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.073914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d30d3-4f2f-4c9d-b143-51c86a34fdea None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.075627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5c31-f6ce-42fa-b60b-bf15225b7a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.076281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5c31-f6ce-42fa-b60b-bf15225b7a28 None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.076560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5c31-f6ce-42fa-b60b-bf15225b7a28 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.076832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a5c31-f6ce-42fa-b60b-bf15225b7a28 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/7d93aa681e2a4427a61fe0bf472c60ec/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.078740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a5c31-f6ce-42fa-b60b-bf15225b7a28 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=7d93aa681e2a4427a61fe0bf472c60ec, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.080172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a5c31-f6ce-42fa-b60b-bf15225b7a28 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.086350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 408/811] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:32.087823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11cf34-effe-4a20-8da3-52a427a16d06 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.088114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11cf34-effe-4a20-8da3-52a427a16d06 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.088392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11cf34-effe-4a20-8da3-52a427a16d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.094063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 404/812] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:31 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/7d93aa681e2a4427a61fe0bf472c60ec/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:32.094914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5c31-f6ce-42fa-b60b-bf15225b7a28 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.107699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-da11cf34-effe-4a20-8da3-52a427a16d06 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:32.108940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-da11cf34-effe-4a20-8da3-52a427a16d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:32.184796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 409/813] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:32.185854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11cf34-effe-4a20-8da3-52a427a16d06 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.244209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a5c31-f6ce-42fa-b60b-bf15225b7a28 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.281756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.307878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.308414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.308700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.308975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.318044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da11cf34-effe-4a20-8da3-52a427a16d06 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.323788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.325337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:32.341305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:32.342599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:32.343717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:32.343717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:32.343717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:32.344698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 405/814] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:32.345599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.395824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.487909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089184a3-a6e1-4086-9c30-3c38e69c0394 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.492849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.493583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.493983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.494331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] PATH_INFO: `/v3/services/network` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.494993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.497510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:32.508005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:32.509272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:36:32.510354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:32.511409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 410/815] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/services/network => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:32.517637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.532590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b83c5-bc23-4a0d-9bc0-944ab7d9dd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.533355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b83c5-bc23-4a0d-9bc0-944ab7d9dd5b None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.533972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b83c5-bc23-4a0d-9bc0-944ab7d9dd5b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.534315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b83c5-bc23-4a0d-9bc0-944ab7d9dd5b None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/803b35616c55468eb7a0f72c9c86b72b/roles/af374fa10d7d4a7aa378572855842eb0` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.558368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b83c5-bc23-4a0d-9bc0-944ab7d9dd5b None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=803b35616c55468eb7a0f72c9c86b72b, role_id=af374fa10d7d4a7aa378572855842eb0)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.560706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b83c5-bc23-4a0d-9bc0-944ab7d9dd5b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.577438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 406/816] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:32 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/803b35616c55468eb7a0f72c9c86b72b/roles/af374fa10d7d4a7aa378572855842eb0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:32.578596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b83c5-bc23-4a0d-9bc0-944ab7d9dd5b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.638790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387d317-a6bb-40e0-af60-1532baa537a2 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.682203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.682772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.683875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.684156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.684432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.685184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:32.685489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.687129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.688880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.708922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 411/817] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:32.709958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.722486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b83c5-bc23-4a0d-9bc0-944ab7d9dd5b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.763756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.764689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.765125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.765543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.766481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] RBAC: Adding query filter params (name=network) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:32.767064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.768608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.774458 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 407/818] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:32.775181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.834989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfb4df-0403-4266-b266-a21f7e8e5278 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.885398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.886065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.886340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.886609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.887771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.889480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.901141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9336d-05f3-4fd5-b4ce-1315d3b6bb3f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:32.901543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:32.902606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:32.903934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 412/819] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:32.905095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.947973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.948610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.948889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.949161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.951429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.958346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.959979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:32.973056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:32.974314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:32.976405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 408/820] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:32.978608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.978987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.037934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a39c2-b71d-4af7-a3f4-9dcea7d7431b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.084980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.085680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.086431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.086719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.087355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] RBAC: Adding query filter params (type=network) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:33.087655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.088675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93beb1-812d-403e-a361-8c6705c76dc7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.089013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.095717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 413/821] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:33.096546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.141254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.142916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.143198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.143472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.144126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:33.144426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.146389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.153840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 409/822] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:33.154663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.213350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57529d-28a4-4465-999d-ebcbaf2d8696 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.226574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.239678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.252591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.253574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.254097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.254593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.255600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:33.256110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.258089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.275537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bcf84-366b-4455-935d-5c33c731136f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.277930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 414/823] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:33.279084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.312717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206971dc-c9d6-4f06-bd68-419649878be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.313395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206971dc-c9d6-4f06-bd68-419649878be3 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.313589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206971dc-c9d6-4f06-bd68-419649878be3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.313892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206971dc-c9d6-4f06-bd68-419649878be3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.314571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206971dc-c9d6-4f06-bd68-419649878be3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.316024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206971dc-c9d6-4f06-bd68-419649878be3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.344788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 410/824] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:33.345680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206971dc-c9d6-4f06-bd68-419649878be3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.418590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f137f56-b647-4c19-a6be-0564dcd1472d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.495899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206971dc-c9d6-4f06-bd68-419649878be3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.509006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.514457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.515260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.515636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.516102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.516838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:33.517238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.518755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.532233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 415/825] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:33.538407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.540736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc83f6-2724-4c65-a9bd-e902e38ceff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.541645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc83f6-2724-4c65-a9bd-e902e38ceff5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.542631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc83f6-2724-4c65-a9bd-e902e38ceff5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.543085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc83f6-2724-4c65-a9bd-e902e38ceff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.633829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc83f6-2724-4c65-a9bd-e902e38ceff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.635929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc83f6-2724-4c65-a9bd-e902e38ceff5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.651458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.745026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eac061-4e85-439b-8c1c-c1f712f07991 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.771609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 411/826] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:36:33.777838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc83f6-2724-4c65-a9bd-e902e38ceff5 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.791879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.810487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a86ccd-2eff-4774-bf79-76581d51df0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.811245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a86ccd-2eff-4774-bf79-76581d51df0b None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.811621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a86ccd-2eff-4774-bf79-76581d51df0b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.812010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a86ccd-2eff-4774-bf79-76581d51df0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.814889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a86ccd-2eff-4774-bf79-76581d51df0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.814889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a86ccd-2eff-4774-bf79-76581d51df0b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.841860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 416/827] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:33.911154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc83f6-2724-4c65-a9bd-e902e38ceff5 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.959275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.959931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.960212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.960549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.972230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.973744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:33.990502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:33.991551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:33.993008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:33.993008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:33.993008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:33.993008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 412/828] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:33.998075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a86ccd-2eff-4774-bf79-76581d51df0b None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.105504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.110358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a86ccd-2eff-4774-bf79-76581d51df0b None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.147044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.147657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.147932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.148201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.148898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:34.149195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.150583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.162469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 417/829] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.168146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.233543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.282667 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dfacb-9d66-451d-8ec4-cd327810b1c3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.321221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.321923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.322213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.322492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.329680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.331167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:34.344349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:34.345448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:34.346408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 413/830] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:34.351363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.471148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b108e-4c3f-461f-b9a6-7761879006e3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.507572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118e0de-6818-4990-83a0-a7ca2b40914d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.508217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e0de-6818-4990-83a0-a7ca2b40914d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.508497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e0de-6818-4990-83a0-a7ca2b40914d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.508769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e0de-6818-4990-83a0-a7ca2b40914d None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.509472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e0de-6818-4990-83a0-a7ca2b40914d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:34.509807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e0de-6818-4990-83a0-a7ca2b40914d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.511242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e0de-6818-4990-83a0-a7ca2b40914d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.525225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 418/831] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.531224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.568895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70acfd-8d46-4c2d-b6ac-9d627c5bb896 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.569215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70acfd-8d46-4c2d-b6ac-9d627c5bb896 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.569449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70acfd-8d46-4c2d-b6ac-9d627c5bb896 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.571842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 419/832] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:34.579893 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ba00-c502-4a4b-9ca0-808b2900ec78 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.580198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ba00-c502-4a4b-9ca0-808b2900ec78 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.580447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63ba00-c502-4a4b-9ca0-808b2900ec78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.604805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5b63ba00-c502-4a4b-9ca0-808b2900ec78 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:34.606885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5b63ba00-c502-4a4b-9ca0-808b2900ec78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:34.625133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.664264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a2eb-422b-4cfe-b90f-11282af657d6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.683885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.697660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 420/833] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:34.719298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61092a-ed9c-4910-bc0a-c07d9c0d432a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.720332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61092a-ed9c-4910-bc0a-c07d9c0d432a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.720795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61092a-ed9c-4910-bc0a-c07d9c0d432a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.721368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61092a-ed9c-4910-bc0a-c07d9c0d432a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.722734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61092a-ed9c-4910-bc0a-c07d9c0d432a None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=2981bb0558d34329a061a5f08149bba3) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:34.723255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61092a-ed9c-4910-bc0a-c07d9c0d432a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.725765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61092a-ed9c-4910-bc0a-c07d9c0d432a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.733340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 414/834] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/role_assignments?user.id=2981bb0558d34329a061a5f08149bba3&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=b698800ceec3460a8519ea627650da0f => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.981961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63ba00-c502-4a4b-9ca0-808b2900ec78 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.055638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b829-288a-4cdf-919a-4a4f24ac3e70 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.055990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b829-288a-4cdf-919a-4a4f24ac3e70 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.056221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b829-288a-4cdf-919a-4a4f24ac3e70 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.057884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 415/835] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:35.063155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dd5be-0a66-429a-9fd2-ecd2b7ae2b2e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.063439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dd5be-0a66-429a-9fd2-ecd2b7ae2b2e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.063710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dd5be-0a66-429a-9fd2-ecd2b7ae2b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.083369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-929dd5be-0a66-429a-9fd2-ecd2b7ae2b2e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:35.084579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-929dd5be-0a66-429a-9fd2-ecd2b7ae2b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:35.093124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63ba00-c502-4a4b-9ca0-808b2900ec78 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.142798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a7009-0f96-4b84-a028-80b10ae38fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.143445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7009-0f96-4b84-a028-80b10ae38fb2 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.144315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7009-0f96-4b84-a028-80b10ae38fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.144594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7009-0f96-4b84-a028-80b10ae38fb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.160797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a7009-0f96-4b84-a028-80b10ae38fb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:35.165785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 416/836] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-208a7009-0f96-4b84-a028-80b10ae38fb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:35.170804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:35.174000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:35.177102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 421/837] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:35.179837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dd5be-0a66-429a-9fd2-ecd2b7ae2b2e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.290476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dd5be-0a66-429a-9fd2-ecd2b7ae2b2e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.305583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:35.329425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.330253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.330418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.330619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.331411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:35.331711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.333073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:35.342575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 417/838] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:35.350715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a7009-0f96-4b84-a028-80b10ae38fb2 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.369093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:35.444469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.463297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a7009-0f96-4b84-a028-80b10ae38fb2 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.512011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.513120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.513645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.514105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.526727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.529444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:35.546461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:35.547457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:35.548358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:35.548358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:35.548358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:35.550461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 422/839] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:35.562323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.598888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bcd9f0-720e-4b5c-956d-669d2b77c7ec None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.644866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:35.649384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc2d7f-f44d-40a1-900e-fef0ea34a8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.650066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc2d7f-f44d-40a1-900e-fef0ea34a8f0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.650350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc2d7f-f44d-40a1-900e-fef0ea34a8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.650620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc2d7f-f44d-40a1-900e-fef0ea34a8f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.667284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc2d7f-f44d-40a1-900e-fef0ea34a8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-78dc2d7f-f44d-40a1-900e-fef0ea34a8f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:35.673989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:35.675047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:35.676001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:35.680414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 418/840] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:35.681570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc2d7f-f44d-40a1-900e-fef0ea34a8f0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.701933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560e15c-c040-4d00-a3a3-4ee6b55870c8 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.731348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:35.741687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.742936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.743557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.744048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.744950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:35.745480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.747422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:35.767881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 423/841] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:35.768700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.823813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc2d7f-f44d-40a1-900e-fef0ea34a8f0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.864634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.865670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.866186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.866676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.868132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:35.868663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.870309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:35.880493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 419/842] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:35.887480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.915266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26a44b-1aad-41cc-910e-90bd381e5a83 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.942289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:35.974214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.975672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.976500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.977270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.987229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.989280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:36.003153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:36.004172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:36:36.005146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:36.012661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 424/843] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:36.013963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.036286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.045399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48e5f-31aa-4b56-8734-aa6a7e80a62a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.095762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.096438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.096727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.097000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.109909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.111484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:36.127490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:36.128517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:36.129425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:36.129425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:36.129425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:36.129425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 420/844] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:36.138004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.150897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27898d37-f0f8-4e3f-be0f-641bb31ba597 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.188636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57eb5e-6c18-4922-905c-5a5550682c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.189664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57eb5e-6c18-4922-905c-5a5550682c4e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.190153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57eb5e-6c18-4922-905c-5a5550682c4e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.190630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57eb5e-6c18-4922-905c-5a5550682c4e None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.191563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57eb5e-6c18-4922-905c-5a5550682c4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:36.192057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57eb5e-6c18-4922-905c-5a5550682c4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.193987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57eb5e-6c18-4922-905c-5a5550682c4e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.213856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 425/845] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:36.215462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479d38c-ce20-4f19-b7e8-d291ffe389d4 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.215810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479d38c-ce20-4f19-b7e8-d291ffe389d4 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.216154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479d38c-ce20-4f19-b7e8-d291ffe389d4 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.217634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 426/846] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:36.222611 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479d38c-ce20-4f19-b7e8-d291ffe389d4 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.251364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a340-e932-4eff-9ba6-1832e2e1a256 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.268901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.289661 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d48c1c-8ecc-4864-8613-25807cbc112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.290355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d48c1c-8ecc-4864-8613-25807cbc112c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.290642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d48c1c-8ecc-4864-8613-25807cbc112c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.290917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d48c1c-8ecc-4864-8613-25807cbc112c None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.291556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d48c1c-8ecc-4864-8613-25807cbc112c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:36.291887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d48c1c-8ecc-4864-8613-25807cbc112c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.293245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d48c1c-8ecc-4864-8613-25807cbc112c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.311664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 421/847] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:36.311664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78569bc-f816-4d0a-a8ed-f7295d170f07 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.311664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78569bc-f816-4d0a-a8ed-f7295d170f07 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.311664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78569bc-f816-4d0a-a8ed-f7295d170f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.329595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-a78569bc-f816-4d0a-a8ed-f7295d170f07 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:36.330877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a78569bc-f816-4d0a-a8ed-f7295d170f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:36.333071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.344075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479d38c-ce20-4f19-b7e8-d291ffe389d4 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.381902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e8caa-9bb2-4f2b-9b28-9805e71b232c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.382513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e8caa-9bb2-4f2b-9b28-9805e71b232c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.382854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e8caa-9bb2-4f2b-9b28-9805e71b232c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.383128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e8caa-9bb2-4f2b-9b28-9805e71b232c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.407762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 422/848] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:36.408674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78569bc-f816-4d0a-a8ed-f7295d170f07 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.479116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e8caa-9bb2-4f2b-9b28-9805e71b232c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.481197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e8caa-9bb2-4f2b-9b28-9805e71b232c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.522568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78569bc-f816-4d0a-a8ed-f7295d170f07 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.551336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.561446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.562112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.562391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.562660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.574430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.577016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.580192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:36.596933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:36.598018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:36.599014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:36.603941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 423/849] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:36.605649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.651808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 427/850] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:36:36.657410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e8caa-9bb2-4f2b-9b28-9805e71b232c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.716453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef34f2-57bd-4804-9bf1-092ff91154fc None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.760267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.760877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.761635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.761923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.762656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:36.762961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.765072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.790312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 424/851] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:36.794514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.798644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e8caa-9bb2-4f2b-9b28-9805e71b232c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.837406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.838064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.838341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.838605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.847684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.849730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.851329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.857136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:36.869209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:36.870571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:36.871662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:36.871662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:36.871662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:36.871662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 428/852] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:36.877188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.910490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22053ca9-1ff9-4c4d-b543-b2af6a749505 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.947892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.948555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.948831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.949098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.949725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.951132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:36.962884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:36.964165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:36.965172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 425/853] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:36.966148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.003745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e068f-ad9b-4909-bf05-e989566ee698 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.043460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4ad2e-6c4c-416b-8449-f3de00d22eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.044313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4ad2e-6c4c-416b-8449-f3de00d22eab None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.044658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4ad2e-6c4c-416b-8449-f3de00d22eab None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.044998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4ad2e-6c4c-416b-8449-f3de00d22eab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.079009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016843e-667b-49b4-a35f-c13d10e7ba79 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.098469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.106021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.113683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.114299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.114583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.114857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.115498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.115801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.116626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4ad2e-6c4c-416b-8449-f3de00d22eab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.117743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.118713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4ad2e-6c4c-416b-8449-f3de00d22eab None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.129563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 426/854] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.130414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.227456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 429/855] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:36:37.228650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033cf48-7d90-46b5-8253-875d035ac2d8 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.228933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033cf48-7d90-46b5-8253-875d035ac2d8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.229208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033cf48-7d90-46b5-8253-875d035ac2d8 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.230731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 430/856] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:37.231261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033cf48-7d90-46b5-8253-875d035ac2d8 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.231926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4950532-d7e3-4668-b1db-8ef8d5222dc9 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.266785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.267393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.267677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.267951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.268579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.268884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.270354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.275001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 427/857] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.275817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.323292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.327463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.338423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033cf48-7d90-46b5-8253-875d035ac2d8 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.373840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c1fbd-056e-4610-8779-906cdf870f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.374469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c1fbd-056e-4610-8779-906cdf870f0f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.374742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c1fbd-056e-4610-8779-906cdf870f0f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.375013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c1fbd-056e-4610-8779-906cdf870f0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.379434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b1049-9405-4293-9451-4e894747b8fe None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.382348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c1fbd-056e-4610-8779-906cdf870f0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.383792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c1fbd-056e-4610-8779-906cdf870f0f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-ee2c1fbd-056e-4610-8779-906cdf870f0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:37.395375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:37.396356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:37.397258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 431/858] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:37.398043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651dae-e176-4785-a728-7a7ab9caa0c7 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.398043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651dae-e176-4785-a728-7a7ab9caa0c7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.398253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37651dae-e176-4785-a728-7a7ab9caa0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.418423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-37651dae-e176-4785-a728-7a7ab9caa0c7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:37.419614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-37651dae-e176-4785-a728-7a7ab9caa0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:37.421273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.421950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.422232 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.422515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.433290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.434782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:37.449934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:37.451052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:37.451958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:37.451958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:37.451958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:37.451958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 428/859] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:37.452195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.492885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 432/860] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:37.493674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651dae-e176-4785-a728-7a7ab9caa0c7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.559669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.570569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86513b35-55de-4340-b15b-63376fda0b7e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.582234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.606834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37651dae-e176-4785-a728-7a7ab9caa0c7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.612295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.612899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.613178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.613448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.614107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.614410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.615784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.625647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 429/861] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.625647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.669730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.670517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.670901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.671274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.672184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.672593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.674212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.689875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 433/862] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.696966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.743656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf40bd-44ec-480b-9083-59294791f4f3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.782114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.782744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.783100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.783378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.784039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.784349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.785787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.801660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 430/863] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.801660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.821553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919caa45-97e7-4c02-821d-2a1c0ac85448 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.831504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.864767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.865381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.865683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.865957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.866685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=7d93aa681e2a4427a61fe0bf472c60ec) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.867054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.868628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.875054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 434/864] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/role_assignments?user.id=7d93aa681e2a4427a61fe0bf472c60ec&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=b698800ceec3460a8519ea627650da0f => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.887069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.901406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.929806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363512ad-8f97-46d7-86dd-8b7c6915bef4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.982283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.983516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.983955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.985041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.985857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.988001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:38.001202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:38.002407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:38.003498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:38.012434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 431/865] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:38.014653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.038120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f625e1-efab-4752-ac97-892c6077fe18 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.079661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.080008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.080241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.080512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.092363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.093864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:38.109292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:38.110339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:38.111258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:38.111258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:38.111258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:38.111258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:38.111258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:38.111258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:38.111258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:38.111258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:38.116206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 435/866] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:38.118693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.157254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.166181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d6363-003b-4562-a09c-2b47ef4ac040 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.202059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.213023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.213999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.213999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.214200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.214846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:38.215155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.216716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.225997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 432/867] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:38.228038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.236301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c7416-e7c5-4f01-b815-849a9f074725 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.285246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.286474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.287123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.287625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.289255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:38.289800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.292179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.309449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 436/868] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:38.315374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.344229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c590e1a-3752-40ac-9575-5fb296fd9a38 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.382025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2497133-f8f3-4978-81da-f7ba16e8d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.383370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497133-f8f3-4978-81da-f7ba16e8d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.383578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497133-f8f3-4978-81da-f7ba16e8d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.383862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2497133-f8f3-4978-81da-f7ba16e8d3d3 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.384611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497133-f8f3-4978-81da-f7ba16e8d3d3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:38.384925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497133-f8f3-4978-81da-f7ba16e8d3d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.386441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2497133-f8f3-4978-81da-f7ba16e8d3d3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.393418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 433/869] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:38.444476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44470892-fcc9-41dc-841f-ac54601bbac2 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.477311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.487617 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfb496a-1dd0-4779-942b-0a53f8428323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.488507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb496a-1dd0-4779-942b-0a53f8428323 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.488974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb496a-1dd0-4779-942b-0a53f8428323 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.489415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb496a-1dd0-4779-942b-0a53f8428323 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.490476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb496a-1dd0-4779-942b-0a53f8428323 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=803b35616c55468eb7a0f72c9c86b72b) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:38.490944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb496a-1dd0-4779-942b-0a53f8428323 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.493171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfb496a-1dd0-4779-942b-0a53f8428323 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.505758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 437/870] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/role_assignments?user.id=803b35616c55468eb7a0f72c9c86b72b&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:39.847520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dfbf9-ecc2-427d-affa-821c085f763f None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.848091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dfbf9-ecc2-427d-affa-821c085f763f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.848091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481dfbf9-ecc2-427d-affa-821c085f763f None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.849671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 434/871] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:39.855221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e1950-b63f-471f-a814-efc5b7c356f1 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.855713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e1950-b63f-471f-a814-efc5b7c356f1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.855793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e1950-b63f-471f-a814-efc5b7c356f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.874966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-394e1950-b63f-471f-a814-efc5b7c356f1 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:39.876159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-394e1950-b63f-471f-a814-efc5b7c356f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:39.949424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 438/872] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:40.209937 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0a315-ed84-4f53-8ba6-af0586f9f3df None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.210281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0a315-ed84-4f53-8ba6-af0586f9f3df None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.210525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b0a315-ed84-4f53-8ba6-af0586f9f3df None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.212715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 435/873] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:40.217965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0a315-ed84-4f53-8ba6-af0586f9f3df None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.219585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08557973-a670-4f47-9802-390505657f3e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.219875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08557973-a670-4f47-9802-390505657f3e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.220249 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08557973-a670-4f47-9802-390505657f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.240136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-08557973-a670-4f47-9802-390505657f3e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:40.241339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-08557973-a670-4f47-9802-390505657f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:40.243338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.343308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 439/874] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:40.359559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b0a315-ed84-4f53-8ba6-af0586f9f3df None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.407279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae3ba3-4331-4919-82ed-822fedd10626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.408250 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae3ba3-4331-4919-82ed-822fedd10626 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.408759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae3ba3-4331-4919-82ed-822fedd10626 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.409242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae3ba3-4331-4919-82ed-822fedd10626 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.410186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae3ba3-4331-4919-82ed-822fedd10626 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.411763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae3ba3-4331-4919-82ed-822fedd10626 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.442021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 436/875] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:40 2022] POST /identity/v3/services => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:40.620720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08557973-a670-4f47-9802-390505657f3e None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.677177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.677440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867947f-0199-47c3-937f-301edaa0195d None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.677651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867947f-0199-47c3-937f-301edaa0195d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.678020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867947f-0199-47c3-937f-301edaa0195d None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.680097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 437/876] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:40.686277 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734a445-d06b-4f26-9931-1bb0a22cc004 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.686546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734a445-d06b-4f26-9931-1bb0a22cc004 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.686825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734a445-d06b-4f26-9931-1bb0a22cc004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.707490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3734a445-d06b-4f26-9931-1bb0a22cc004 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:40.711335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3734a445-d06b-4f26-9931-1bb0a22cc004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:40.738872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08557973-a670-4f47-9802-390505657f3e None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.792797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a79410-2406-42f7-b573-fb968f073d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.793436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79410-2406-42f7-b573-fb968f073d8e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.793763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79410-2406-42f7-b573-fb968f073d8e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.794033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a79410-2406-42f7-b573-fb968f073d8e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.794616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a79410-2406-42f7-b573-fb968f073d8e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.796014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a79410-2406-42f7-b573-fb968f073d8e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.800633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 438/877] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-60a79410-2406-42f7-b573-fb968f073d8e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:40.805576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:40.806819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:40.807888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 440/878] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:40.812758 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734a445-d06b-4f26-9931-1bb0a22cc004 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.816603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526bdeb-8aae-4447-9687-6306613dc270 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.816875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526bdeb-8aae-4447-9687-6306613dc270 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.817152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526bdeb-8aae-4447-9687-6306613dc270 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.818720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.818830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 441/879] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:40.827056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1933-d28d-4eee-aa3c-b92769527c47 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.827349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1933-d28d-4eee-aa3c-b92769527c47 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.827616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f1933-d28d-4eee-aa3c-b92769527c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.858719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-719f1933-d28d-4eee-aa3c-b92769527c47 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:40.859962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-719f1933-d28d-4eee-aa3c-b92769527c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:40.938395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734a445-d06b-4f26-9931-1bb0a22cc004 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.959715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 442/880] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:40.982963 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.983916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.984345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.984760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.985579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:40.986205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.987796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.993839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 439/881] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.999126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f1933-d28d-4eee-aa3c-b92769527c47 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.058456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.109364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.125387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f1933-d28d-4eee-aa3c-b92769527c47 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.166857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.167930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.169897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.170390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.171221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:41.171721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.173277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.178739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 443/882] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:41.229984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.245875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.278238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609db031-3848-48b7-9dfb-2f42c945ccd5 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.320337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88a36a-c37d-43bb-a104-de507b9c08f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.321271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88a36a-c37d-43bb-a104-de507b9c08f3 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.321767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88a36a-c37d-43bb-a104-de507b9c08f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.322271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe88a36a-c37d-43bb-a104-de507b9c08f3 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.323154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88a36a-c37d-43bb-a104-de507b9c08f3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.324733 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe88a36a-c37d-43bb-a104-de507b9c08f3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.352269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 440/883] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:41 2022] POST /identity/v3/services => generated 269 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:41.363991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab27af2-56b5-454c-a461-4dbb63c429dd None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.367438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.417998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dc416-5898-4701-ad6e-3c96b5d53425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.418664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dc416-5898-4701-ad6e-3c96b5d53425 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.418938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dc416-5898-4701-ad6e-3c96b5d53425 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.419209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7dc416-5898-4701-ad6e-3c96b5d53425 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.433198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7dc416-5898-4701-ad6e-3c96b5d53425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-cf7dc416-5898-4701-ad6e-3c96b5d53425 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:41.438264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:41.439277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:36:41.440137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:41.441829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 444/884] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:41.446642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88a36a-c37d-43bb-a104-de507b9c08f3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.576494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe88a36a-c37d-43bb-a104-de507b9c08f3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.615210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.626871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.627833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.628660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.629337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.630328 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:41.630844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.633061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.642402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 441/885] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:41.650962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dc416-5898-4701-ad6e-3c96b5d53425 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.764922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7dc416-5898-4701-ad6e-3c96b5d53425 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.806423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.806423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.806423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.806423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.817758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.819669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:41.839332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:41.840491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:41.841714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:41.841714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:41.841714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:41.843472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 445/886] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:41.851847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.956053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40cabb-f7d1-4b92-9c65-376b54f6b3c9 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.994898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.995510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.995795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.996075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.996722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:41.997026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.998603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:42.011666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 442/887] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:42.023515 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:42.040177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.138448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24394-5704-4580-bbaf-a4cc49087600 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.174256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.174953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.175244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.175516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.184213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:42.185733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:42.198217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:42.200413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:42.202942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 446/888] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:36:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:42.210719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:42.270779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.332230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb062fb-38f2-41d3-8b7c-6ee7e26bbbcf None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.377754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.378568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.378880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.379162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.379903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:42.380215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:42.381744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:42.400882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 443/889] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 13 19:36:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:42.406734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:42.489569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.521843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb39e37-02d2-4c9e-ba6f-9cb299f9cddb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.557455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb30d4d-7b42-443f-90b8-4cd249e4f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.558241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb30d4d-7b42-443f-90b8-4cd249e4f2c5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.558523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb30d4d-7b42-443f-90b8-4cd249e4f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.558795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb30d4d-7b42-443f-90b8-4cd249e4f2c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.629280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb30d4d-7b42-443f-90b8-4cd249e4f2c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:42.630781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb30d4d-7b42-443f-90b8-4cd249e4f2c5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:42.709013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.742315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 447/890] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:42 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:36:42.750935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:42.869067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfee00-b1c4-4d2a-9ffe-47bf38993a92 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.877008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.904624 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f5a4-4531-481b-b616-f7d21aa79428 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.904932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f5a4-4531-481b-b616-f7d21aa79428 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.905172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749f5a4-4531-481b-b616-f7d21aa79428 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.906788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 448/891] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:42.908865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.909803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.910295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.910794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.915578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91b77e-76da-4966-9a40-09e830d03eda None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.915984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91b77e-76da-4966-9a40-09e830d03eda None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.916395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91b77e-76da-4966-9a40-09e830d03eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.925354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:42.927209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:42.936583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fa91b77e-76da-4966-9a40-09e830d03eda None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:42.938559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fa91b77e-76da-4966-9a40-09e830d03eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:42.944210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:42.945533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:42.946550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:42.946550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:42.946550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:42.948791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 444/892] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:42.954232 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.025813 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 449/893] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:43.080896 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31593727-18bd-41d3-a210-eb1b8a3c6998 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.116498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.117413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.117890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.118305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.119107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.119552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.121132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.139251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 445/894] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:43.140830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91b77e-76da-4966-9a40-09e830d03eda None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.268423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91b77e-76da-4966-9a40-09e830d03eda None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.271076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.309883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.310519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.310791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.311057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.318086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.319525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.331140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:43.333584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:43.334533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:43.335533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:43.335533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:43.335533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:43.335533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:43.335533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:43.335533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:43.335533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:43.335533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:43.341059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 450/895] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:43.342451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.368546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.482278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03550870-6823-4aa2-b3d9-011503840789 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.504824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186776a-d4f4-42dd-abc0-60eb5f5161a6 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.531378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.531929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.532235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.532583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.533281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.533671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.535512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.547769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f0b3a-0d8d-4243-aa2f-0f65cccfbea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.548510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0b3a-0d8d-4243-aa2f-0f65cccfbea7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.548791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0b3a-0d8d-4243-aa2f-0f65cccfbea7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.549056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f0b3a-0d8d-4243-aa2f-0f65cccfbea7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.549673 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f0b3a-0d8d-4243-aa2f-0f65cccfbea7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.551065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f0b3a-0d8d-4243-aa2f-0f65cccfbea7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.551937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 451/896] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:43.558403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-cf4f0b3a-0d8d-4243-aa2f-0f65cccfbea7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:43.565859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:43.567877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:43.569054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 446/897] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:43.572164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc44e9-65c9-42fa-ade3-185b373c542b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.572446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc44e9-65c9-42fa-ade3-185b373c542b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.573056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc44e9-65c9-42fa-ade3-185b373c542b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.574403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 447/898] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:43.575194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc44e9-65c9-42fa-ade3-185b373c542b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.591937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.685449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.696494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde4db9-a04d-4491-b9be-a28420d7c7e4 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.718848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc44e9-65c9-42fa-ade3-185b373c542b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.747092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5288fa-ed8d-4613-b5de-b664d647c1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.747719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5288fa-ed8d-4613-b5de-b664d647c1b3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.747994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5288fa-ed8d-4613-b5de-b664d647c1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.748261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5288fa-ed8d-4613-b5de-b664d647c1b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.749002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5288fa-ed8d-4613-b5de-b664d647c1b3 None admin] RBAC: Adding query filter params (role.id=719de673a5ee4eb8a7e528f62ec5603d, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=803b35616c55468eb7a0f72c9c86b72b) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.749319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5288fa-ed8d-4613-b5de-b664d647c1b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.751449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5288fa-ed8d-4613-b5de-b664d647c1b3 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.757661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 452/899] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/role_assignments?user.id=803b35616c55468eb7a0f72c9c86b72b&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=719de673a5ee4eb8a7e528f62ec5603d => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:43.758946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d0a1-cd6c-4edc-ab7d-811e3c4271d8 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.759325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d0a1-cd6c-4edc-ab7d-811e3c4271d8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.759602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266d0a1-cd6c-4edc-ab7d-811e3c4271d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.771255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.772276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.772760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.773874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.774710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.775215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.776815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.779904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-7266d0a1-cd6c-4edc-ab7d-811e3c4271d8 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:43.781566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7266d0a1-cd6c-4edc-ab7d-811e3c4271d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:43.787559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 448/900] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:43.789880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.875002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 453/901] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:43.876646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684eb46-9547-41b1-8b80-04e9a37ff688 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.877077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684eb46-9547-41b1-8b80-04e9a37ff688 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.877518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684eb46-9547-41b1-8b80-04e9a37ff688 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.879007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 454/902] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:43.885370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f1222-7ed2-4e0c-b154-0cbcb05787ed None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.885696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f1222-7ed2-4e0c-b154-0cbcb05787ed None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.885968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f1222-7ed2-4e0c-b154-0cbcb05787ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.907630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-463f1222-7ed2-4e0c-b154-0cbcb05787ed None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:43.908853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-463f1222-7ed2-4e0c-b154-0cbcb05787ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:43.921551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.925158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ad2ea-35ce-45ae-a80e-1bd627f46ac5 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.983922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.985204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.985770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.986257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.986988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 455/903] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:43.987380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.987792 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.989205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.997639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 449/904] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.001451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f1222-7ed2-4e0c-b154-0cbcb05787ed None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.149686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f1222-7ed2-4e0c-b154-0cbcb05787ed None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.173562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.187495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.191839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.192478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.192751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.193016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.194316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] RBAC: Adding query filter params (interface=public, service_id=32ce65a1a04e45d2a6784301505e721e, region_id=RegionOne) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:44.194649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.197477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.209676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 456/905] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/endpoints?service_id=32ce65a1a04e45d2a6784301505e721e&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.212579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.225116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.305015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f20e99-eb25-4bc5-be29-06abf9c4014c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.317421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b824303-2031-456c-bbf8-6725c792f5fb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.342406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c109f5f-61fb-427c-9a30-3493a06bd812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.343024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c109f5f-61fb-427c-9a30-3493a06bd812 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.343309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c109f5f-61fb-427c-9a30-3493a06bd812 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.343587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c109f5f-61fb-427c-9a30-3493a06bd812 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.344245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c109f5f-61fb-427c-9a30-3493a06bd812 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.345646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c109f5f-61fb-427c-9a30-3493a06bd812 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.356497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c9d0a-7e9a-4292-9644-44b00d477ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.357162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c9d0a-7e9a-4292-9644-44b00d477ab5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.357449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c9d0a-7e9a-4292-9644-44b00d477ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.357740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c9d0a-7e9a-4292-9644-44b00d477ab5 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.358342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c9d0a-7e9a-4292-9644-44b00d477ab5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.359777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c9d0a-7e9a-4292-9644-44b00d477ab5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.361420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 450/906] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:44 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:44.362181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c109f5f-61fb-427c-9a30-3493a06bd812 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.378078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 457/907] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:36:44.455011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.504716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c109f5f-61fb-427c-9a30-3493a06bd812 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.547131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.548288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.548790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.549278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.550175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.551972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:44.562518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:44.563576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:44.564497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:44.567603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 451/908] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:44.574617 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c9d0a-7e9a-4292-9644-44b00d477ab5 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.665715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.692390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c9d0a-7e9a-4292-9644-44b00d477ab5 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.733260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.733846 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.734125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.734400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.735635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:44.736199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.737703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.743851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 458/909] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.750077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.860716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fc751-d103-44e1-8043-2f1e8f4bc8d0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.892428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.899264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.899921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.900203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.900474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.901154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:44.901461 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.902947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.914144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 452/910] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.916639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.037700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d28bc-c345-4f65-9f1e-11fdb6a4f35e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.075048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd6c75-efdb-46ce-bd59-e5ce4b85036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.076012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6c75-efdb-46ce-bd59-e5ce4b85036a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.076513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6c75-efdb-46ce-bd59-e5ce4b85036a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.077014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd6c75-efdb-46ce-bd59-e5ce4b85036a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.077952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd6c75-efdb-46ce-bd59-e5ce4b85036a None admin] RBAC: Adding query filter params (interface=public, service_id=505cc3de8cb24d34b46d7840f6e0152f, region_id=RegionOne) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:45.078474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd6c75-efdb-46ce-bd59-e5ce4b85036a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.080016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd6c75-efdb-46ce-bd59-e5ce4b85036a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.086159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 459/911] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/endpoints?service_id=505cc3de8cb24d34b46d7840f6e0152f&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:45.093989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.202388 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd66873c-a436-4872-a66d-87789025bd26 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.248218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842fd52-3528-41ca-b6dd-29ce4c9da383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.248894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842fd52-3528-41ca-b6dd-29ce4c9da383 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.249184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842fd52-3528-41ca-b6dd-29ce4c9da383 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.249460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842fd52-3528-41ca-b6dd-29ce4c9da383 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.250123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842fd52-3528-41ca-b6dd-29ce4c9da383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.251551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842fd52-3528-41ca-b6dd-29ce4c9da383 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.259021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 453/912] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:36:46.189396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb90c-d273-4db6-ab59-3e5a0ddfd4b2 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.189958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb90c-d273-4db6-ab59-3e5a0ddfd4b2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.189958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fb90c-d273-4db6-ab59-3e5a0ddfd4b2 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.191443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 460/913] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:46.197422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b64b-bd1f-4a5c-8e2b-32d72bcfd143 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.198024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b64b-bd1f-4a5c-8e2b-32d72bcfd143 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.198505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f7b64b-bd1f-4a5c-8e2b-32d72bcfd143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.219839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-94f7b64b-bd1f-4a5c-8e2b-32d72bcfd143 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:46.222650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-94f7b64b-bd1f-4a5c-8e2b-32d72bcfd143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:46.292410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.307365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 454/914] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:46.585852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fb90c-d273-4db6-ab59-3e5a0ddfd4b2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.589146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.706044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fb90c-d273-4db6-ab59-3e5a0ddfd4b2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.728954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6551-aebf-411d-b17c-6aad04cf910a None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.729941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6551-aebf-411d-b17c-6aad04cf910a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.730513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b6551-aebf-411d-b17c-6aad04cf910a None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.733236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 455/915] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:46.740278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e48976-a5b2-44cb-b8ab-09288a3ef16b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.740565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e48976-a5b2-44cb-b8ab-09288a3ef16b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.740839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e48976-a5b2-44cb-b8ab-09288a3ef16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.749155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affac6e3-7095-458b-be8f-30eec87f2942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.750967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affac6e3-7095-458b-be8f-30eec87f2942 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.751504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affac6e3-7095-458b-be8f-30eec87f2942 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.751988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affac6e3-7095-458b-be8f-30eec87f2942 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.762362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-04e48976-a5b2-44cb-b8ab-09288a3ef16b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:46.764111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-04e48976-a5b2-44cb-b8ab-09288a3ef16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:46.764553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affac6e3-7095-458b-be8f-30eec87f2942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-affac6e3-7095-458b-be8f-30eec87f2942 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:46.769475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:46.770632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:36:46.771694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:46.773455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 461/916] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:46.781651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affac6e3-7095-458b-be8f-30eec87f2942 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.847746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 456/917] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:46.849317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c637546-5120-497f-9b1b-c4c9eaac2242 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.849978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c637546-5120-497f-9b1b-c4c9eaac2242 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.849978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c637546-5120-497f-9b1b-c4c9eaac2242 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.851829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 457/918] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:46.853277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.857539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2533-709f-4d25-b35e-32c971579b04 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.857918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2533-709f-4d25-b35e-32c971579b04 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.858255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f2533-709f-4d25-b35e-32c971579b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.877347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-281f2533-709f-4d25-b35e-32c971579b04 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:46.878840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-281f2533-709f-4d25-b35e-32c971579b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:46.897909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affac6e3-7095-458b-be8f-30eec87f2942 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.938256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.939123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.939586 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.940010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.940856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:46.941329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.942975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.957231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 462/919] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:46.958489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.994269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 458/920] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:47.066729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e92ef-6a4e-430d-ad67-739cc70d8b5a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.100056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.103716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.104418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.104847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.105182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.116986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.118590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:47.134032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:47.135023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:47.135948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:47.135948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:47.135948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:47.135948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 463/921] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:47.141078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f2533-709f-4d25-b35e-32c971579b04 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.195350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.251656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f2533-709f-4d25-b35e-32c971579b04 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.269133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.293970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.294586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.294868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.295140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.295785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:47.296088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.297710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.310475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 459/922] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:47.311369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.380104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed2a994-a7f2-4c0e-a44e-16cec35733a1 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.419969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.420710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.420984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.421276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.421971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.423322 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.427097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17146488-96ea-4efa-9690-18e178ad2ad8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:47.432967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:47.434756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:36:47.436590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:47.439289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 464/923] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:47.440254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.442738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.478970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.479924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.480367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.480648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.481234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.482648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:47.492070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:47.493406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:47.498185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 460/924] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:47.499010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.501647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.578967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b7c38-733e-4887-9b43-d32425f4fb93 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.605822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0613af-b5c0-4c90-8ddd-0f17a28d61ca None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.618281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.619340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.619803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.620088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.629819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.631390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:47.647466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:47.649206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 465/925] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:47.650629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.652226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f5ad6-720a-47c4-a5cd-60a2f386e38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.652847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f5ad6-720a-47c4-a5cd-60a2f386e38b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.653199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f5ad6-720a-47c4-a5cd-60a2f386e38b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.653392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f5ad6-720a-47c4-a5cd-60a2f386e38b None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.654064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f5ad6-720a-47c4-a5cd-60a2f386e38b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:47.654368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f5ad6-720a-47c4-a5cd-60a2f386e38b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.655749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f5ad6-720a-47c4-a5cd-60a2f386e38b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.660790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 461/926] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:47.662021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6020f-7dc4-46f3-a471-0409375a1c3a None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.662308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6020f-7dc4-46f3-a471-0409375a1c3a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.662582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6020f-7dc4-46f3-a471-0409375a1c3a None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.663841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 462/927] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:47.665642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6020f-7dc4-46f3-a471-0409375a1c3a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.702371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.755281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.758888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de993f81-a51a-42c1-9a30-fc2f78c991b4 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.769214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6020f-7dc4-46f3-a471-0409375a1c3a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.793335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.793986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.794267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.794544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.795178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:47.795501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.796867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.801597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 466/928] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:47.802441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.802677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f7400-f373-4605-abaf-46b9e64dde82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.803303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7400-f373-4605-abaf-46b9e64dde82 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.803565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7400-f373-4605-abaf-46b9e64dde82 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.803863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f7400-f373-4605-abaf-46b9e64dde82 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.804607 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f7400-f373-4605-abaf-46b9e64dde82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:47.804889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f7400-f373-4605-abaf-46b9e64dde82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.806523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f7400-f373-4605-abaf-46b9e64dde82 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.824512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 463/929] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:47.825920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1199ba-08be-49e5-a3ab-bdd3a4a33488 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.826203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1199ba-08be-49e5-a3ab-bdd3a4a33488 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.826473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1199ba-08be-49e5-a3ab-bdd3a4a33488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.852741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-dd1199ba-08be-49e5-a3ab-bdd3a4a33488 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:47.854060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dd1199ba-08be-49e5-a3ab-bdd3a4a33488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:47.907195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa907e4-a11c-4331-b9bf-06d75d8fecbb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.921799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 464/930] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:47.922612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1199ba-08be-49e5-a3ab-bdd3a4a33488 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.941932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.942541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.942823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.943089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.943710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:47.944010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.945749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.951426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 467/931] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:47.952801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.956616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.985379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.048140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1199ba-08be-49e5-a3ab-bdd3a4a33488 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.069164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524594df-ca61-4165-8dcf-085697349e2e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.083200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.083831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.084115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.084394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.085538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.085894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.087273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.097982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 465/932] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.097982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.111561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.112431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.112576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.112741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.123698 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.125250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:48.139764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:48.140757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:48.142278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:48.142278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:48.142278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:48.142278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 468/933] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:48.142278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.195456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.213683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4a540-d222-4786-bea0-e52b0218e0bf None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.232166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.251913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff10ea-c42e-4a83-830a-fdc664915fdc None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.253763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b829d6b-4187-4219-a955-fd724430e2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.254386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829d6b-4187-4219-a955-fd724430e2be None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.254660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829d6b-4187-4219-a955-fd724430e2be None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.254931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829d6b-4187-4219-a955-fd724430e2be None admin] PATH_INFO: `/v3/endpoints` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.255596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b829d6b-4187-4219-a955-fd724430e2be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.257036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b829d6b-4187-4219-a955-fd724430e2be None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.277006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 466/934] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:36:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:48.277957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b829d6b-4187-4219-a955-fd724430e2be None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.304202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.306038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.306559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.307020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.307931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] RBAC: Adding query filter params (interface=public, service_id=cabb0073481e4ba3b0967b50f72cf60d, region_id=RegionOne) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.309132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.311278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.320687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 469/935] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/endpoints?service_id=cabb0073481e4ba3b0967b50f72cf60d&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.322147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.402856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b829d6b-4187-4219-a955-fd724430e2be None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.442979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.443592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.443870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.444136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.445350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358ac51-d850-4d44-a538-7975c7edd978 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.445530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.445746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.447118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.459886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 467/936] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.465273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.467264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.499937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.500603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.500936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.501818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.502433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.504676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:48.514513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:48.515570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:48.516491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 470/937] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:48.517860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.523507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.583823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123f7b3-cd07-423c-97a6-4d6c1f6bd263 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.621261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd493a35-6659-4ea6-8fc3-27036c61afaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.621908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd493a35-6659-4ea6-8fc3-27036c61afaf None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.622192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd493a35-6659-4ea6-8fc3-27036c61afaf None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.622467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd493a35-6659-4ea6-8fc3-27036c61afaf None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.623074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd493a35-6659-4ea6-8fc3-27036c61afaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.624449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd493a35-6659-4ea6-8fc3-27036c61afaf None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.630485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1bc07-a3fd-49c9-b546-29e7bf1f6701 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.631071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 468/938] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:36:48.631900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd493a35-6659-4ea6-8fc3-27036c61afaf None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.684665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.685318 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.685597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.685941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.694413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.696253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:48.716291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:48.717272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:48.718223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:48.718223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:48.718223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:48.718223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:48.718223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:48.718223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:48.718223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:48.718223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:48.719311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 471/939] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:48.729303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.730050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.779077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd493a35-6659-4ea6-8fc3-27036c61afaf None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.816496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.822241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.823621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.823990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.824334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.825334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.825793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.827504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.837945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 469/940] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.839937 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.873635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247d828-a79c-4b9d-b120-ddc4411c23d9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.906236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.906944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.907327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.907609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.908307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.908614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.910085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.924917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 472/941] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.930924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.946949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052495db-a95d-45c6-a690-f8d579b87c24 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.978791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.985270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.985998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.986428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.986991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.987787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.988263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.989798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.996597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 470/942] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:49.002185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.046256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dbac0-613f-4b06-b8e7-606487f3df1e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.074188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.086981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d9a6d-1d8f-4b0c-ac61-b0c720987899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.087772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d9a6d-1d8f-4b0c-ac61-b0c720987899 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.088060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d9a6d-1d8f-4b0c-ac61-b0c720987899 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.088337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d9a6d-1d8f-4b0c-ac61-b0c720987899 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/803b35616c55468eb7a0f72c9c86b72b/roles/719de673a5ee4eb8a7e528f62ec5603d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.090711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d9a6d-1d8f-4b0c-ac61-b0c720987899 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=803b35616c55468eb7a0f72c9c86b72b, role_id=719de673a5ee4eb8a7e528f62ec5603d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.092276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d9a6d-1d8f-4b0c-ac61-b0c720987899 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.114100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7057880-bdc5-4dac-b902-e3619c829158 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.119320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 473/943] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:48 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/803b35616c55468eb7a0f72c9c86b72b/roles/719de673a5ee4eb8a7e528f62ec5603d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:49.155681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49feb714-f5b2-402f-aed1-adc50dfb4e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.156672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feb714-f5b2-402f-aed1-adc50dfb4e85 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.157183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feb714-f5b2-402f-aed1-adc50dfb4e85 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.157662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49feb714-f5b2-402f-aed1-adc50dfb4e85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.158529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49feb714-f5b2-402f-aed1-adc50dfb4e85 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.160137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49feb714-f5b2-402f-aed1-adc50dfb4e85 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.186340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 471/944] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:36:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:50.548441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba34eb-9b09-475c-8785-2188edfba791 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.548970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba34eb-9b09-475c-8785-2188edfba791 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.548970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba34eb-9b09-475c-8785-2188edfba791 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.550604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 474/945] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:50.556542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300bc45-2d31-45f7-81e0-8372f8bc70d9 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.556953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300bc45-2d31-45f7-81e0-8372f8bc70d9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.557123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300bc45-2d31-45f7-81e0-8372f8bc70d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.576495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c300bc45-2d31-45f7-81e0-8372f8bc70d9 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:50.577749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c300bc45-2d31-45f7-81e0-8372f8bc70d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:50.628639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.657165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 472/946] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:50.907516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8891333-9c57-4472-a991-22746da94f5f None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.907871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8891333-9c57-4472-a991-22746da94f5f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.908071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8891333-9c57-4472-a991-22746da94f5f None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.909430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 475/947] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:50.915419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b506cd-bfe9-40ab-b76b-ee5ebcd8d350 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.915825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b506cd-bfe9-40ab-b76b-ee5ebcd8d350 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.916206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b506cd-bfe9-40ab-b76b-ee5ebcd8d350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.934652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-d5b506cd-bfe9-40ab-b76b-ee5ebcd8d350 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:50.935843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d5b506cd-bfe9-40ab-b76b-ee5ebcd8d350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:51.010171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 473/948] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:51.160952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8891333-9c57-4472-a991-22746da94f5f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.302369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b506cd-bfe9-40ab-b76b-ee5ebcd8d350 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.344165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8891333-9c57-4472-a991-22746da94f5f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.384494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.385995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.386743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.387199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.388185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.389869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.392685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING keystone.server.flask.common [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 19:36:51.411999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.sql.core [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:36:51.411999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 19:36:51.411999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [parameters: {'id': 'd016f177b1c84821964f4cc55d647924', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 19:36:51.411999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:36:51.421287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b506cd-bfe9-40ab-b76b-ee5ebcd8d350 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-9ee93c62-3b81-455c-b174-ed055dd82808 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d016f177b1c84821964f4cc55d647924.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d016f177b1c84821964f4cc55d647924. Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:36:51.429583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result.read() Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 19:36:51.430612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application yield resource Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise self.value Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application session.commit() Sep 13 19:36:51.431883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:36:51.433034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:36:51.434873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result.read() Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:36:51.436273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [parameters: {'id': 'd016f177b1c84821964f4cc55d647924', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:51.437431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:51.438457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d016f177b1c84821964f4cc55d647924. Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 476/949] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Sep 13 19:36:51 2022] POST /identity/v3/projects => generated 262 bytes in 274 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355e20-47ff-4110-92b1-76ac22834b1c None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.439654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355e20-47ff-4110-92b1-76ac22834b1c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.440927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed355e20-47ff-4110-92b1-76ac22834b1c None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.441018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.441106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 477/950] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:51.442090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb61623-2567-4f9f-8399-5bbac90c76bc None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.442266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb61623-2567-4f9f-8399-5bbac90c76bc None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.442456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb61623-2567-4f9f-8399-5bbac90c76bc None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.443772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 478/951] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:51.444940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb61623-2567-4f9f-8399-5bbac90c76bc None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.471145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d92f8-3e23-496d-9fc0-db63df804589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.472021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d92f8-3e23-496d-9fc0-db63df804589 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.472263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d92f8-3e23-496d-9fc0-db63df804589 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.472841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d92f8-3e23-496d-9fc0-db63df804589 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.473458 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d92f8-3e23-496d-9fc0-db63df804589 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.475089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d92f8-3e23-496d-9fc0-db63df804589 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-f42d92f8-3e23-496d-9fc0-db63df804589 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:51.484824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:51.486059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:51.487325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 474/952] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:51.491131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.492140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e423db-0d90-4760-b8b1-da3364fa6206 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.492435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e423db-0d90-4760-b8b1-da3364fa6206 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.492741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e423db-0d90-4760-b8b1-da3364fa6206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.510960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-31e423db-0d90-4760-b8b1-da3364fa6206 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:51.512159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-31e423db-0d90-4760-b8b1-da3364fa6206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:51.552150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb61623-2567-4f9f-8399-5bbac90c76bc None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.579931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 475/953] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:51.581308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec8f60-bf5e-44c6-811b-4a839dcfa3e3 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.581594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec8f60-bf5e-44c6-811b-4a839dcfa3e3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.581894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec8f60-bf5e-44c6-811b-4a839dcfa3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.593962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.594603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.594888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.595165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.602370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-afec8f60-bf5e-44c6-811b-4a839dcfa3e3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:51.602993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.603700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-afec8f60-bf5e-44c6-811b-4a839dcfa3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:51.604581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:51.618930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:51.619954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 479/954] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:51.620910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.677715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 476/955] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:51.678396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec8f60-bf5e-44c6-811b-4a839dcfa3e3 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.711904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.752402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.756918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8c6f0-8c87-4211-982a-e287cff4c1b3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.794574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec8f60-bf5e-44c6-811b-4a839dcfa3e3 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.797401 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.798041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.798323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.798594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.799221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:51.799523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.800887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.810890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 480/956] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:51.811774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.842816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.843436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.844110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.844397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.845067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:51.845369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.846876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.864520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 477/957] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:51.865350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.937458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd32646-38eb-49a5-82bc-5cdeec3ca5ed None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.977304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.978017 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.978300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.978695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.979349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:51.979653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.981167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.987315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 481/958] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:51.988324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.996337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.007217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f157f2-eb5a-40cb-b7b9-34bcfa0fbfa2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.036834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.046237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdab14a-e4db-4e56-97a4-f6872e1c71a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.046895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab14a-e4db-4e56-97a4-f6872e1c71a8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.047181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab14a-e4db-4e56-97a4-f6872e1c71a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.047460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab14a-e4db-4e56-97a4-f6872e1c71a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.060051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdab14a-e4db-4e56-97a4-f6872e1c71a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-7bdab14a-e4db-4e56-97a4-f6872e1c71a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:52.064076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:52.065042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 478/959] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:52.066044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdab14a-e4db-4e56-97a4-f6872e1c71a8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.132742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823e825-9295-49df-8091-4f0316528f1f None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.169111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.170199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.170485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.170777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.171332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.173176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.181925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdab14a-e4db-4e56-97a4-f6872e1c71a8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:52.182550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:52.183604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:52.184496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 482/960] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:52.185307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.228381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256074a-446e-43c9-8ed4-a7d6c69a5ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.228908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256074a-446e-43c9-8ed4-a7d6c69a5ee9 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.230012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256074a-446e-43c9-8ed4-a7d6c69a5ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.230296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256074a-446e-43c9-8ed4-a7d6c69a5ee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.230985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256074a-446e-43c9-8ed4-a7d6c69a5ee9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.232402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256074a-446e-43c9-8ed4-a7d6c69a5ee9 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.253266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 479/961] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:36:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:52.259050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256074a-446e-43c9-8ed4-a7d6c69a5ee9 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.276822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.305853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.317684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbeec4f-76e5-4e8f-b7ac-e4a59de28e24 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.392693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.394028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.394513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.395257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.396706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:52.397473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.399963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.406986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256074a-446e-43c9-8ed4-a7d6c69a5ee9 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.418861 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 483/962] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:52.419493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.455602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.456584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.457010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.457287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.458091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] RBAC: Adding query filter params (name=compute) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:52.458285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.460301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.466067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 480/963] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:52.472857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.556446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bc185-dd32-49c6-95a6-b2175deb0041 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.564185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.596214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.597204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.597583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.597993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.606478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfadc07-2791-4c12-b8a3-cf801aaefbbf None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.611460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.612848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:52.629244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:52.630319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:52.631234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:52.631234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:52.631234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:52.631234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 484/964] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:52.638794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.641954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.655911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.657108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.657722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.658541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.659953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:52.660682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.663170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.669991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 481/965] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:52.768604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a47346-0ab8-4164-821e-7f0fe483f518 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.808271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.808869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.809155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.809430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.810125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:52.810428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.811919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.826560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 485/966] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:52.831907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.943686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb203f7a-f5a0-44bf-9fcf-9a99f392cfa7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.979463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.980095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.980372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.980640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.983607 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.990381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.992003 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:53.003980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:53.004991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:53.006039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 482/967] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:53.015025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.088248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.129775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ede0b0-a039-4c29-9372-a25dc4ae482c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.166706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.167325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.167608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.167879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.168614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:53.168922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.170383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.189637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 486/968] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:53.195150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.303164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6893a-9269-4f9e-8685-12299e635c83 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.341096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c925a1b-46d2-4bce-a571-eb803451f334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.342105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c925a1b-46d2-4bce-a571-eb803451f334 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.342587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c925a1b-46d2-4bce-a571-eb803451f334 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.343074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c925a1b-46d2-4bce-a571-eb803451f334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.405498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.414018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c925a1b-46d2-4bce-a571-eb803451f334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.415784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c925a1b-46d2-4bce-a571-eb803451f334 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.517911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 483/969] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:36:53.529656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.634065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d1cc-3ea7-463f-947d-114f7d2f00fc None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.655644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.671687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.672402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.672902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.673423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.686715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.688186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:53.707102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:53.708112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:53.709028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:53.709028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:53.709028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:53.709028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 487/970] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:53.716080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c925a1b-46d2-4bce-a571-eb803451f334 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.814565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.823268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c925a1b-46d2-4bce-a571-eb803451f334 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.858716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.859703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.860181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.860664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.861496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:53.862020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.863575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.875541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 484/971] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:53.881269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.988610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83dc0a-e610-425f-9f76-254d302e9562 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.026244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df519fd3-5433-42d4-821a-093f5e35f99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.026244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df519fd3-5433-42d4-821a-093f5e35f99e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.026244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df519fd3-5433-42d4-821a-093f5e35f99e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.026244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df519fd3-5433-42d4-821a-093f5e35f99e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.033293 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df519fd3-5433-42d4-821a-093f5e35f99e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.034794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df519fd3-5433-42d4-821a-093f5e35f99e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-df519fd3-5433-42d4-821a-093f5e35f99e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:36:54.047433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:54.048510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:54.050105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 488/972] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:54.054698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.179781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d8411-aa1d-42e0-8f65-1e386e5ae563 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.215754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.216373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.216657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.216939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.217732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:54.218044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.219482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.232236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 485/973] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:54.233551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664206-e3cf-490c-8c1f-6a9fbf76f4ab None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.233877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664206-e3cf-490c-8c1f-6a9fbf76f4ab None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.234152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664206-e3cf-490c-8c1f-6a9fbf76f4ab None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.235570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 489/974] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:54.242377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.243719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fd73-a8ee-4ba9-82fc-c110106f47b0 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.244228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fd73-a8ee-4ba9-82fc-c110106f47b0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.244707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41fd73-a8ee-4ba9-82fc-c110106f47b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.261202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.268253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-0e41fd73-a8ee-4ba9-82fc-c110106f47b0 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:54.270115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0e41fd73-a8ee-4ba9-82fc-c110106f47b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:54.301767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.365684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314424e-b52d-4425-8c91-ce1ae2aced88 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.366577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 490/975] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:54.401166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac92eba-f915-4639-9384-519bbc3fd497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.401928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92eba-f915-4639-9384-519bbc3fd497 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.402249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92eba-f915-4639-9384-519bbc3fd497 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.402525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac92eba-f915-4639-9384-519bbc3fd497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.403263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92eba-f915-4639-9384-519bbc3fd497 None admin] RBAC: Adding query filter params (role.id=719de673a5ee4eb8a7e528f62ec5603d, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=803b35616c55468eb7a0f72c9c86b72b) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:54.403572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92eba-f915-4639-9384-519bbc3fd497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.405053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac92eba-f915-4639-9384-519bbc3fd497 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.412401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 486/976] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/role_assignments?user.id=803b35616c55468eb7a0f72c9c86b72b&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=719de673a5ee4eb8a7e528f62ec5603d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:54.547460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5f3be-0922-4800-9c5f-196b118e3c81 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.547652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5f3be-0922-4800-9c5f-196b118e3c81 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.547904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5f3be-0922-4800-9c5f-196b118e3c81 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.549736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 491/977] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:54.557048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091c1b5-08e6-4f0d-8f94-2930dc315e08 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.557048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091c1b5-08e6-4f0d-8f94-2930dc315e08 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.557921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091c1b5-08e6-4f0d-8f94-2930dc315e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.580531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-a091c1b5-08e6-4f0d-8f94-2930dc315e08 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:54.582894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a091c1b5-08e6-4f0d-8f94-2930dc315e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:54.639241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5f3be-0922-4800-9c5f-196b118e3c81 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.676366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 487/978] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:54.760351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5f3be-0922-4800-9c5f-196b118e3c81 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.799896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.801201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.801836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.802291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.803134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.804818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.821185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.sql.core [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:36:54.821185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 19:36:54.821185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [parameters: {'user_id': '7f5930f0eb4648798ff45f828508d44e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 19:36:54.821185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result.read() Sep 13 19:36:54.830277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:36:54.831362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application yield resource Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise self.value Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application session.commit() Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:36:54.832328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:36:54.833277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:36:54.834261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application result.read() Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:36:54.835718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [parameters: {'user_id': '7f5930f0eb4648798ff45f828508d44e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:54.836552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 19:36:54.837356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 19:36:54.838707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 19:36:54.838707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 19:36:54.838707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:36:54.838707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:36:54.838707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 19:36:54.838707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:54.840794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 492/979] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Sep 13 19:36:54 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:54.849671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091c1b5-08e6-4f0d-8f94-2930dc315e08 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.860848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.968858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091c1b5-08e6-4f0d-8f94-2930dc315e08 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.974074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.005075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.014762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64419e-d2f9-4212-b0f4-41f9e48f4500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.017984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64419e-d2f9-4212-b0f4-41f9e48f4500 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.017984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64419e-d2f9-4212-b0f4-41f9e48f4500 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.017984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64419e-d2f9-4212-b0f4-41f9e48f4500 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.030166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64419e-d2f9-4212-b0f4-41f9e48f4500 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.031865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64419e-d2f9-4212-b0f4-41f9e48f4500 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-fd64419e-d2f9-4212-b0f4-41f9e48f4500 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:36:55.047778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:55.048804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:55.049787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 488/980] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:55.050970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0b810-778d-4f48-9627-4e3654e12620 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.050970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0b810-778d-4f48-9627-4e3654e12620 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.051133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0b810-778d-4f48-9627-4e3654e12620 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.052701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 489/981] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:55.057990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df955a2-5bc4-4022-9c17-66fcf105a596 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.058281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df955a2-5bc4-4022-9c17-66fcf105a596 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.058561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df955a2-5bc4-4022-9c17-66fcf105a596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.078489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7df955a2-5bc4-4022-9c17-66fcf105a596 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:55.079753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7df955a2-5bc4-4022-9c17-66fcf105a596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:55.100855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496054b9-69e3-4600-ba3d-218dbde4d1be None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.138668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.139327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.139598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.139867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.140455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.142165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.150363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:55.151262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:55.152421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:55.153320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 493/982] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:55.154271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.162054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 490/983] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:55.163709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df955a2-5bc4-4022-9c17-66fcf105a596 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.279013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.295945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430adda8-2814-4f89-81b4-a231c080cb04 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.296677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df955a2-5bc4-4022-9c17-66fcf105a596 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.343223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.343699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.344069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.344408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.345116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:55.345491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.346991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.349592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.351530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.352177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.352906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.357684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 491/984] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:55.361347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:55.361970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.362643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.364656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.433257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 494/985] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:55.468051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.470381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.518029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b5945-788b-4592-af1d-43a3458bda79 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.574210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.575083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.575583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.576059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.577878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:55.578682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.581351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.589714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 492/986] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:55.592434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8532e-26d4-4141-b16d-b3f09919f727 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.595335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.637524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15a189-7ade-45e2-a928-c1c5a816cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.638588 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15a189-7ade-45e2-a928-c1c5a816cc18 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.639096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15a189-7ade-45e2-a928-c1c5a816cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.639575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15a189-7ade-45e2-a928-c1c5a816cc18 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.640888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15a189-7ade-45e2-a928-c1c5a816cc18 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.643227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15a189-7ade-45e2-a928-c1c5a816cc18 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.663029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 495/987] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:55 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:55.722886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f9305-8be3-487b-b8a1-f27fcdf1ef06 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.756130 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.774581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85134ade-3bc4-4528-ac15-ff90626eefa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.775208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134ade-3bc4-4528-ac15-ff90626eefa7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.775488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134ade-3bc4-4528-ac15-ff90626eefa7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.775761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85134ade-3bc4-4528-ac15-ff90626eefa7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.776549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85134ade-3bc4-4528-ac15-ff90626eefa7 None admin] RBAC: Adding query filter params (interface=admin, service_id=cabb0073481e4ba3b0967b50f72cf60d, region_id=RegionOne) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:55.776865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85134ade-3bc4-4528-ac15-ff90626eefa7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.778352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85134ade-3bc4-4528-ac15-ff90626eefa7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.787480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 493/988] 10.222.0.23 () {56 vars in 1249 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/endpoints?service_id=cabb0073481e4ba3b0967b50f72cf60d&interface=admin®ion_id=RegionOne => generated 193 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:55.794106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15a189-7ade-45e2-a928-c1c5a816cc18 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.804369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.909871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15a189-7ade-45e2-a928-c1c5a816cc18 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.946288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5593ae5-fa93-42b9-9fa8-647f68d59f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.946904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5593ae5-fa93-42b9-9fa8-647f68d59f4c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.947183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5593ae5-fa93-42b9-9fa8-647f68d59f4c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.947452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5593ae5-fa93-42b9-9fa8-647f68d59f4c None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.948058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5593ae5-fa93-42b9-9fa8-647f68d59f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.949425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5593ae5-fa93-42b9-9fa8-647f68d59f4c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.961654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 496/989] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:36:56.846114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f581cd-0273-429e-a4c7-166238650e29 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.846701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f581cd-0273-429e-a4c7-166238650e29 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.846701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f581cd-0273-429e-a4c7-166238650e29 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.848264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 494/990] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:56.854240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570bf8d-61d0-4526-b580-dead8b7767d1 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.854686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570bf8d-61d0-4526-b580-dead8b7767d1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.854763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570bf8d-61d0-4526-b580-dead8b7767d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.873414 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f570bf8d-61d0-4526-b580-dead8b7767d1 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:56.874953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f570bf8d-61d0-4526-b580-dead8b7767d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:56.943100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:56.957641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 497/991] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:57.243736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f581cd-0273-429e-a4c7-166238650e29 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.365685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f581cd-0273-429e-a4c7-166238650e29 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.408006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.408644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.408917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.409217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] PATH_INFO: `/v3/services/image` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.409813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:57.411178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:57.423179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:57.424422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:36:57.425302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:57.430197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 495/992] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Sep 13 19:36:57 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:57.431208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570bf8d-61d0-4526-b580-dead8b7767d1 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.546268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570bf8d-61d0-4526-b580-dead8b7767d1 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.582962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cf444-5f83-4161-9b6b-d02c20bac34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.583587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cf444-5f83-4161-9b6b-d02c20bac34c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.583865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cf444-5f83-4161-9b6b-d02c20bac34c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.584135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cf444-5f83-4161-9b6b-d02c20bac34c None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.584829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cf444-5f83-4161-9b6b-d02c20bac34c None admin] RBAC: Adding query filter params (name=image) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:57.585132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cf444-5f83-4161-9b6b-d02c20bac34c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:57.586736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cf444-5f83-4161-9b6b-d02c20bac34c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:57.596700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 498/993] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:57.597698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.661817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:57.712796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c08f1-9a09-4ef7-87b0-a5f787e05628 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.749542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d28560-e937-47ba-9009-bc4f470e261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.750186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28560-e937-47ba-9009-bc4f470e261b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.750469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28560-e937-47ba-9009-bc4f470e261b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.750742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d28560-e937-47ba-9009-bc4f470e261b None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.751374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d28560-e937-47ba-9009-bc4f470e261b None admin] RBAC: Adding query filter params (type=image) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:57.751677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d28560-e937-47ba-9009-bc4f470e261b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:57.753052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d28560-e937-47ba-9009-bc4f470e261b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:57.765649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 496/994] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:57.929114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645231e-9231-48b3-a530-88ee3f63d1ad None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.929767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645231e-9231-48b3-a530-88ee3f63d1ad None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.929767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645231e-9231-48b3-a530-88ee3f63d1ad None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.931326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 499/995] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:57.937636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca4b15-a26d-45bf-a476-cb7a68ac7df8 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.938127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca4b15-a26d-45bf-a476-cb7a68ac7df8 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.938219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca4b15-a26d-45bf-a476-cb7a68ac7df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.957691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-faca4b15-a26d-45bf-a476-cb7a68ac7df8 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:57.961243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-faca4b15-a26d-45bf-a476-cb7a68ac7df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:58.048040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 497/996] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:36:58.123459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95ec3-91f7-41e3-a57d-4e7573606689 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.123804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95ec3-91f7-41e3-a57d-4e7573606689 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.124010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95ec3-91f7-41e3-a57d-4e7573606689 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.125496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 500/997] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:58.131204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe083de-b9a2-4799-b320-26b4a2f794a0 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.131204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe083de-b9a2-4799-b320-26b4a2f794a0 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.132003 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe083de-b9a2-4799-b320-26b4a2f794a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.151969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-9fe083de-b9a2-4799-b320-26b4a2f794a0 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:58.153164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9fe083de-b9a2-4799-b320-26b4a2f794a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:58.207154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.244139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 498/998] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:36:58.330548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c95ec3-91f7-41e3-a57d-4e7573606689 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.405942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.475842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c95ec3-91f7-41e3-a57d-4e7573606689 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.511031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe083de-b9a2-4799-b320-26b4a2f794a0 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.537991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.538636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.538908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.539289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.539886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.541259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:58.554169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:58.555252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:36:58.556108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:36:58.556973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 501/999] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:58.561695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.628185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe083de-b9a2-4799-b320-26b4a2f794a0 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.667692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.668365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.668648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.668922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.669541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.671034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:58.681738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:58.685916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:36:58.686968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 499/1000] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:58.687931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc0289-9a4a-43af-ae23-22377517d8a2 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.688862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.719377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.729124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.729699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.729980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.730253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.730890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:58.731220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.732551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.737469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.743294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 502/1001] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:58.744966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.800461 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfad7a-bb3e-4b85-a486-d902157d0ce7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.836695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.837301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.837577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.837876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.838507 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:58.838813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.840194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.845092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 500/1002] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.851246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.879103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b400ae-6add-45b0-b4e9-64cd51b8ef91 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.925237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.926193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.926626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.927053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.927931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] RBAC: Adding query filter params (type=compute) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:58.928983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.931477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.938847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 503/1003] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:58.944755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.972446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a90d5-4499-49ae-b217-944b94d57154 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.974857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.019459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.020094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.020377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.020649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.021291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:59.021593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.024006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.030549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.033569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 501/1004] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:59.040243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.099253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba52a1-cf5c-4988-a7bf-2d2ccb85d3fb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.153325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.154024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.154328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.154643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.155396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] RBAC: Adding query filter params (interface=public, service_id=43e474c8d34b41f1a51f34d247c1b746, region_id=RegionOne) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:59.155753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.157286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.163827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 504/1005] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/endpoints?service_id=43e474c8d34b41f1a51f34d247c1b746&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:59.169897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.176249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23442fac-b0e7-49b0-ac00-5de9a031b29b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.218695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00168cb7-ae62-4ca1-97cb-65c03bef13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.219327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168cb7-ae62-4ca1-97cb-65c03bef13a5 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.219657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168cb7-ae62-4ca1-97cb-65c03bef13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.219940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00168cb7-ae62-4ca1-97cb-65c03bef13a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.220711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00168cb7-ae62-4ca1-97cb-65c03bef13a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.222153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00168cb7-ae62-4ca1-97cb-65c03bef13a5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.243201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 502/1006] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:36:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:59.309079 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133d754-bee7-4695-9854-d39b177bc166 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.332124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.367371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fcbb3-8914-4b74-93e4-26d48ff445c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.367995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcbb3-8914-4b74-93e4-26d48ff445c4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.368270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcbb3-8914-4b74-93e4-26d48ff445c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.368542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcbb3-8914-4b74-93e4-26d48ff445c4 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.369152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fcbb3-8914-4b74-93e4-26d48ff445c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.370573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fcbb3-8914-4b74-93e4-26d48ff445c4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.377689 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 505/1007] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/services => generated 1727 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:37:00.028645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32e749-41e5-4252-8f23-c079f99d19ac None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.029132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32e749-41e5-4252-8f23-c079f99d19ac None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.029228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32e749-41e5-4252-8f23-c079f99d19ac None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.030609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 503/1008] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:00.035944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582d6f3-9c32-45d5-978b-d7259f5547e8 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.036371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582d6f3-9c32-45d5-978b-d7259f5547e8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.036557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582d6f3-9c32-45d5-978b-d7259f5547e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.055418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-b582d6f3-9c32-45d5-978b-d7259f5547e8 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:00.056617 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b582d6f3-9c32-45d5-978b-d7259f5547e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:00.130415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 506/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:00.400203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32e749-41e5-4252-8f23-c079f99d19ac None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.432122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.522685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32e749-41e5-4252-8f23-c079f99d19ac None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.561845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549b2145-0396-41d9-b40b-10dbb6c6a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.562829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b2145-0396-41d9-b40b-10dbb6c6a97a None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.563316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b2145-0396-41d9-b40b-10dbb6c6a97a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.563800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549b2145-0396-41d9-b40b-10dbb6c6a97a None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.564668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b2145-0396-41d9-b40b-10dbb6c6a97a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.566290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549b2145-0396-41d9-b40b-10dbb6c6a97a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.589753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 504/1010] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:00 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:01.483683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9f847-f081-439e-854a-3113d561ec66 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.484274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9f847-f081-439e-854a-3113d561ec66 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.484274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9f847-f081-439e-854a-3113d561ec66 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.486136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 507/1011] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:01.492654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b70c-de3d-4375-bc87-416d30948d83 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.493201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b70c-de3d-4375-bc87-416d30948d83 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.493201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1b70c-de3d-4375-bc87-416d30948d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.512580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-6ff1b70c-de3d-4375-bc87-416d30948d83 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:01.513796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6ff1b70c-de3d-4375-bc87-416d30948d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:01.588825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b537f-2c4e-4e24-afee-ae46b632369b None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.589112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b537f-2c4e-4e24-afee-ae46b632369b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.589384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b537f-2c4e-4e24-afee-ae46b632369b None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.590892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 508/1012] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:01.595615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e6e8-4225-4595-bc9c-77041f45059b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.595905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e6e8-4225-4595-bc9c-77041f45059b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.596183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e6e8-4225-4595-bc9c-77041f45059b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.599054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 505/1013] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:01 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:01.614609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5106e6e8-4225-4595-bc9c-77041f45059b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:01.616394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5106e6e8-4225-4595-bc9c-77041f45059b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:01.684502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 509/1014] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:01 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:01.872871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1b70c-de3d-4375-bc87-416d30948d83 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.888970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.958126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106e6e8-4225-4595-bc9c-77041f45059b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.989944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1b70c-de3d-4375-bc87-416d30948d83 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.008840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.035595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef1bac-d139-44ed-846d-0360ac27bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.036394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef1bac-d139-44ed-846d-0360ac27bd3d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.036975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef1bac-d139-44ed-846d-0360ac27bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.037257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef1bac-d139-44ed-846d-0360ac27bd3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.040729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef1bac-d139-44ed-846d-0360ac27bd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.042216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef1bac-d139-44ed-846d-0360ac27bd3d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.044011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 506/1015] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.049538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef1bac-d139-44ed-846d-0360ac27bd3d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.084275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106e6e8-4225-4595-bc9c-77041f45059b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.134044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.134605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.134883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.135157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] PATH_INFO: `/v3/services/compute` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.135742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.137117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.141284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:02.147902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:02.148947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:02.149837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 510/1016] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:02.154764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.166725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef1bac-d139-44ed-846d-0360ac27bd3d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.204493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a849f16-21f2-4b2b-8f6a-f12730de641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.205103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849f16-21f2-4b2b-8f6a-f12730de641f None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.205381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849f16-21f2-4b2b-8f6a-f12730de641f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.205676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849f16-21f2-4b2b-8f6a-f12730de641f None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.206340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a849f16-21f2-4b2b-8f6a-f12730de641f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.207779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a849f16-21f2-4b2b-8f6a-f12730de641f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.236933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 507/1017] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:02 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:02.277795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b3791-c29f-400f-af09-f68f1d27230a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.283453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.326076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.327041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.327544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.328058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.329066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:02.329535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.332015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.345107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 511/1018] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:02.347366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a849f16-21f2-4b2b-8f6a-f12730de641f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.472797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a849f16-21f2-4b2b-8f6a-f12730de641f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.511016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bcd57-954f-45c5-ab73-8aafdb5b79cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.511654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bcd57-954f-45c5-ab73-8aafdb5b79cf None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.511895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bcd57-954f-45c5-ab73-8aafdb5b79cf None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.512165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bcd57-954f-45c5-ab73-8aafdb5b79cf None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.512803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bcd57-954f-45c5-ab73-8aafdb5b79cf None admin] RBAC: Adding query filter params (type=compute) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:02.513103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bcd57-954f-45c5-ab73-8aafdb5b79cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.514544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bcd57-954f-45c5-ab73-8aafdb5b79cf None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.516133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.521045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 508/1019] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:02.527031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.637001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e90361-979d-4fd1-81e9-1b58603443d3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.673002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d08cef8-912f-43c5-b898-c943f15b9be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.673639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cef8-912f-43c5-b898-c943f15b9be8 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.674001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cef8-912f-43c5-b898-c943f15b9be8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.674283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d08cef8-912f-43c5-b898-c943f15b9be8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.674978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08cef8-912f-43c5-b898-c943f15b9be8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.676351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d08cef8-912f-43c5-b898-c943f15b9be8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.704579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 512/1020] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:37:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:02.842300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a630-2c13-42af-95fa-cea97dc74475 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.842613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a630-2c13-42af-95fa-cea97dc74475 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.842777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63a630-2c13-42af-95fa-cea97dc74475 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.844393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 509/1021] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:02.850396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ea561-9f1f-49f2-bf6f-e052d2de51d0 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.850741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ea561-9f1f-49f2-bf6f-e052d2de51d0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.850974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ea561-9f1f-49f2-bf6f-e052d2de51d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.870814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-2d2ea561-9f1f-49f2-bf6f-e052d2de51d0 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:02.872072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2d2ea561-9f1f-49f2-bf6f-e052d2de51d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:02.898779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.970110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 513/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:03.420557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63a630-2c13-42af-95fa-cea97dc74475 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.526405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63a630-2c13-42af-95fa-cea97dc74475 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.563957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.564598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.564883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.565151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] PATH_INFO: `/v3/services/image` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.565770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.567155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:03.577494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:03.578639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:03.582830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 510/1023] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:03.585790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ea561-9f1f-49f2-bf6f-e052d2de51d0 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.697750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ea561-9f1f-49f2-bf6f-e052d2de51d0 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.733375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.734031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.734309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.734577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.735238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] RBAC: Adding query filter params (name=image) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.735539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.736911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.748203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 514/1024] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:03.750084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.812840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.854297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eb494-d8d6-45a6-800f-97fb02f17591 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.891034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.891639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.891923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.892189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.892809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] RBAC: Adding query filter params (type=image) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.893109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.894496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.899522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 511/1025] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:03.905410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.924652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:04.016842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c272c-c65a-4ef4-a58f-990b91b42ccb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.053435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db759ae2-9d21-4a2f-a51f-65b7134bd14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.054090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db759ae2-9d21-4a2f-a51f-65b7134bd14a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.054369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db759ae2-9d21-4a2f-a51f-65b7134bd14a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.054644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db759ae2-9d21-4a2f-a51f-65b7134bd14a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.055359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db759ae2-9d21-4a2f-a51f-65b7134bd14a None admin] RBAC: Adding query filter params (interface=public, service_id=d4e2565ef09b412dbfa9e88604f987b3, region_id=RegionOne) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:04.055667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db759ae2-9d21-4a2f-a51f-65b7134bd14a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:04.057037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db759ae2-9d21-4a2f-a51f-65b7134bd14a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:04.071500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 515/1026] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/endpoints?service_id=d4e2565ef09b412dbfa9e88604f987b3&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:04.072863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:04.189941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472156a3-4999-4dcf-86e2-7a0c93c1626d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.218438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:04.229007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031bbfd5-86e2-4027-bfd4-8f5ea057f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.229647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bbfd5-86e2-4027-bfd4-8f5ea057f8cf None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.229935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bbfd5-86e2-4027-bfd4-8f5ea057f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.230213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bbfd5-86e2-4027-bfd4-8f5ea057f8cf None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.230848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bbfd5-86e2-4027-bfd4-8f5ea057f8cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:04.232224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bbfd5-86e2-4027-bfd4-8f5ea057f8cf None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:04.239040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 512/1027] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:37:04 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:37:04.592461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5a24-bb01-4492-9324-96c31c3d5821 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.592996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5a24-bb01-4492-9324-96c31c3d5821 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.592996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5a24-bb01-4492-9324-96c31c3d5821 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.594552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 516/1028] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:04.599849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef9131-da6b-4f6d-b873-f5db8ce20543 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.600326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef9131-da6b-4f6d-b873-f5db8ce20543 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.600470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef9131-da6b-4f6d-b873-f5db8ce20543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.619079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-0bef9131-da6b-4f6d-b873-f5db8ce20543 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:04.620269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0bef9131-da6b-4f6d-b873-f5db8ce20543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:04.693568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 513/1029] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:04.965832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a5a24-bb01-4492-9324-96c31c3d5821 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.058631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb3e4-801c-4f99-8947-2676cd367a36 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.059123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb3e4-801c-4f99-8947-2676cd367a36 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.059590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb3e4-801c-4f99-8947-2676cd367a36 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.061097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 514/1030] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:05.067014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdab12-9e23-4285-839c-4dc67f737182 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.067294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdab12-9e23-4285-839c-4dc67f737182 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.067586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdab12-9e23-4285-839c-4dc67f737182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.072870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.087118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2dcdab12-9e23-4285-839c-4dc67f737182 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:05.088989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2dcdab12-9e23-4285-839c-4dc67f737182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:05.117529 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a5a24-bb01-4492-9324-96c31c3d5821 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.165803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 515/1031] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:05.166026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3bd0f-1605-4a8a-85d7-54fd65a1a9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.166956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3bd0f-1605-4a8a-85d7-54fd65a1a9fb None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.167367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3bd0f-1605-4a8a-85d7-54fd65a1a9fb None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.167649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3bd0f-1605-4a8a-85d7-54fd65a1a9fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.168842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3bd0f-1605-4a8a-85d7-54fd65a1a9fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.170327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3bd0f-1605-4a8a-85d7-54fd65a1a9fb None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.172161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 517/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:05.177731 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdab12-9e23-4285-839c-4dc67f737182 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.238674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.317107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdab12-9e23-4285-839c-4dc67f737182 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.354670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b411c79-417f-4009-a37b-51d90bf44750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.355276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b411c79-417f-4009-a37b-51d90bf44750 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.355581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b411c79-417f-4009-a37b-51d90bf44750 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.355858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b411c79-417f-4009-a37b-51d90bf44750 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.356464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b411c79-417f-4009-a37b-51d90bf44750 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.357928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b411c79-417f-4009-a37b-51d90bf44750 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.383638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 516/1033] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:05 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:05.448052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3bd0f-1605-4a8a-85d7-54fd65a1a9fb None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.567417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3bd0f-1605-4a8a-85d7-54fd65a1a9fb None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.612578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b1f13-fb01-4a24-ba4e-fb3a5ae182a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.613461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b1f13-fb01-4a24-ba4e-fb3a5ae182a3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.613781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b1f13-fb01-4a24-ba4e-fb3a5ae182a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.614034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b1f13-fb01-4a24-ba4e-fb3a5ae182a3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.628298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70b1f13-fb01-4a24-ba4e-fb3a5ae182a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-f70b1f13-fb01-4a24-ba4e-fb3a5ae182a3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:05.634647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:05.635644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:05.636506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 518/1034] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:05.642583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b411c79-417f-4009-a37b-51d90bf44750 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.766285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b411c79-417f-4009-a37b-51d90bf44750 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.774282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.823904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.824706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.825020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.825498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.826694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:05.827251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.828813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.840135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 517/1035] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:05.846581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b1f13-fb01-4a24-ba4e-fb3a5ae182a3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.858288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.968147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b1f13-fb01-4a24-ba4e-fb3a5ae182a3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.013311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.013311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.013311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.013731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.032036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.033503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:06.049725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:06.050853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:06.052059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:06.052059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:06.052059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:06.053748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 519/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:06.057202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.175357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67776b9f-76fd-40c1-9da7-49012d75685d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.214288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.214898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.215185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.215457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.216094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:06.216400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.217790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:06.233804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 518/1037] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:06.237648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.308989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.363640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37962ded-f65d-4d5b-bcec-b85a51b79533 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.405053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62353d5-aaeb-46f6-8fda-3bc33e2cea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.405720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62353d5-aaeb-46f6-8fda-3bc33e2cea8b None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.406002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62353d5-aaeb-46f6-8fda-3bc33e2cea8b None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.406278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62353d5-aaeb-46f6-8fda-3bc33e2cea8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.414974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62353d5-aaeb-46f6-8fda-3bc33e2cea8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.416410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62353d5-aaeb-46f6-8fda-3bc33e2cea8b None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-c62353d5-aaeb-46f6-8fda-3bc33e2cea8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:06.427925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:06.429036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:06.430171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:06.431631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 520/1038] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:06.435045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.441833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1e21a-7efd-4456-b87e-b55e8916d9b7 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.442120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1e21a-7efd-4456-b87e-b55e8916d9b7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.442479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1e21a-7efd-4456-b87e-b55e8916d9b7 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.444649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 521/1039] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:06.449543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c05cb3-a020-4b6b-bc41-d1a253c6d195 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.449857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c05cb3-a020-4b6b-bc41-d1a253c6d195 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.450131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c05cb3-a020-4b6b-bc41-d1a253c6d195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.469434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-18c05cb3-a020-4b6b-bc41-d1a253c6d195 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:06.470929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-18c05cb3-a020-4b6b-bc41-d1a253c6d195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:06.473766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.540766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 522/1040] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:06.567184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3c316-6565-424a-8b74-26f6bd9c050f None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.604849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.605789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.606219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.606652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.607812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:06.608309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.610701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:06.642883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 519/1041] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:06.644825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c05cb3-a020-4b6b-bc41-d1a253c6d195 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.702848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.759793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c05cb3-a020-4b6b-bc41-d1a253c6d195 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.794697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332e97c-97cb-4182-83a1-01dd30475797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.795368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332e97c-97cb-4182-83a1-01dd30475797 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.795649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332e97c-97cb-4182-83a1-01dd30475797 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.795925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332e97c-97cb-4182-83a1-01dd30475797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.816081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.885916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332e97c-97cb-4182-83a1-01dd30475797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.887372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332e97c-97cb-4182-83a1-01dd30475797 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:06.935468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.939662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.955152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bdf47-3716-4754-afa2-5800f34cf243 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.991340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.991978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.992260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.992532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] PATH_INFO: `/v3/services/image` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.993112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.994607 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:07.004569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:07.007065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:07.014099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 520/1042] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:07.014928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 523/1043] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:07.015005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.015091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332e97c-97cb-4182-83a1-01dd30475797 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.129214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffcf1-b408-4595-b20a-fd5e3fe19726 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.132551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332e97c-97cb-4182-83a1-01dd30475797 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.181733 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.182652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.182946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.183098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.183397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.183652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.183817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.184213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.184385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] RBAC: Adding query filter params (name=image) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.184552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.186530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.195721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 521/1044] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.204683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.206595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.206772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.218490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:07.227902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:07.229820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:07.232234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:07.232234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:07.232234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:07.233657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.234170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 524/1045] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:07.235296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.336271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb76bf-e5a1-4126-a2a4-040954bf6c52 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.365416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15a600-5cf1-4ce9-ba27-7c7c7a8b065e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.385315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.385968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.386257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.386537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.387175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] RBAC: Adding query filter params (type=image) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.387486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.388869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.394614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 522/1046] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.400620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.415766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.416391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.416665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.417216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.417675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.417988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.419348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.437770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 525/1047] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.440728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.494555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.506028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.523682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f376beb4-ae92-47d5-bf3f-70e4382b3436 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.551592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1655444-e936-4a94-8711-54c5472a94d2 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.559284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e198-63a2-4a89-907f-4dd4ed3e714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.560023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e198-63a2-4a89-907f-4dd4ed3e714e None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.560363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e198-63a2-4a89-907f-4dd4ed3e714e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.560570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e198-63a2-4a89-907f-4dd4ed3e714e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.561225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037e198-63a2-4a89-907f-4dd4ed3e714e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.562738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037e198-63a2-4a89-907f-4dd4ed3e714e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.588378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 523/1048] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:37:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:07.604594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb6c77-3e17-413b-8a02-caf70aaf109e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.605414 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb6c77-3e17-413b-8a02-caf70aaf109e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.605862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb6c77-3e17-413b-8a02-caf70aaf109e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.606262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb6c77-3e17-413b-8a02-caf70aaf109e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.615051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb6c77-3e17-413b-8a02-caf70aaf109e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.616774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb6c77-3e17-413b-8a02-caf70aaf109e None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-8acb6c77-3e17-413b-8a02-caf70aaf109e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:07.630119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:07.631827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:07.633068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:07.633068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:07.633068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:07.633068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:07.633068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:07.633068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:07.633068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:07.633068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:07.634236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 526/1049] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:07.639573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e198-63a2-4a89-907f-4dd4ed3e714e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.718213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c55f97-91a7-4d2b-8ca5-a5d0085dcde7 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.718975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c55f97-91a7-4d2b-8ca5-a5d0085dcde7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.718975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c55f97-91a7-4d2b-8ca5-a5d0085dcde7 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.720369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 527/1050] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:07.726731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d64a1-90ce-49dd-ac7f-078ddfe1faa5 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.727051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d64a1-90ce-49dd-ac7f-078ddfe1faa5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.727288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d64a1-90ce-49dd-ac7f-078ddfe1faa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.748224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-940d64a1-90ce-49dd-ac7f-078ddfe1faa5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:07.749388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-940d64a1-90ce-49dd-ac7f-078ddfe1faa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:07.781001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e198-63a2-4a89-907f-4dd4ed3e714e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.830155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.830770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.831052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.831322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.833134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.833134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.833784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.836154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.849441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 528/1051] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:07.855885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 524/1052] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.862069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d64a1-90ce-49dd-ac7f-078ddfe1faa5 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.912478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.977119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d64a1-90ce-49dd-ac7f-078ddfe1faa5 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.013202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.013773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.014051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.014330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.017296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] RBAC: Adding query filter params (role.id=719de673a5ee4eb8a7e528f62ec5603d, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=2981bb0558d34329a061a5f08149bba3) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:08.017296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.017296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.024903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 529/1053] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/role_assignments?user.id=2981bb0558d34329a061a5f08149bba3&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=719de673a5ee4eb8a7e528f62ec5603d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.125692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.245454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a36706-e21a-451b-a375-7a49758b47d7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.284593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbd732-b507-408e-89a0-e59e40a22955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.285158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd732-b507-408e-89a0-e59e40a22955 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.285440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd732-b507-408e-89a0-e59e40a22955 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.285744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbd732-b507-408e-89a0-e59e40a22955 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.303038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbd732-b507-408e-89a0-e59e40a22955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-94bbd732-b507-408e-89a0-e59e40a22955 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:08.308932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.310051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:08.310990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:08.313669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 525/1054] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:08.317049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.437498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf25f14-9896-4293-ac0b-19daa652f85c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.448779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.476519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.477445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.477927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.478408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.479336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:08.479835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.481430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.496432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 530/1055] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.497789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbd732-b507-408e-89a0-e59e40a22955 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.567125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.615365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbd732-b507-408e-89a0-e59e40a22955 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.657821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10d006-e227-4af5-b97a-1146ae6948e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.658768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10d006-e227-4af5-b97a-1146ae6948e7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.659056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10d006-e227-4af5-b97a-1146ae6948e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.659524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10d006-e227-4af5-b97a-1146ae6948e7 None admin] PATH_INFO: `/v3/users/a551c780c77945cebb37fee2d66a72aa` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.680727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10d006-e227-4af5-b97a-1146ae6948e7 None admin] RBAC: Authorizing `identity:get_user(user_id=a551c780c77945cebb37fee2d66a72aa)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.682752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10d006-e227-4af5-b97a-1146ae6948e7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.689494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 526/1056] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/users/a551c780c77945cebb37fee2d66a72aa => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.691265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.804872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0a6ec-2b75-44fe-9abb-c5838af970e6 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.842657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba061c1d-b88a-4ee7-8ac3-89043d11dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.843244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba061c1d-b88a-4ee7-8ac3-89043d11dc8a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.843526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba061c1d-b88a-4ee7-8ac3-89043d11dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.843797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba061c1d-b88a-4ee7-8ac3-89043d11dc8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.888798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.922286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba061c1d-b88a-4ee7-8ac3-89043d11dc8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.923794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba061c1d-b88a-4ee7-8ac3-89043d11dc8a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.096618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 531/1057] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:09.103166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10d006-e227-4af5-b97a-1146ae6948e7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.213642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10d006-e227-4af5-b97a-1146ae6948e7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.238992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.276806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68a47f-85a7-4c44-9390-a138cc25476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.277913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68a47f-85a7-4c44-9390-a138cc25476b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.278416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68a47f-85a7-4c44-9390-a138cc25476b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.278911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68a47f-85a7-4c44-9390-a138cc25476b None admin] PATH_INFO: `/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.280856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68a47f-85a7-4c44-9390-a138cc25476b None admin] RBAC: Authorizing `identity:get_project(project_id=f38e6571879e43c3a43f7378fb4a0ce1)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.283614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68a47f-85a7-4c44-9390-a138cc25476b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.287767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 527/1058] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:09.299102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba061c1d-b88a-4ee7-8ac3-89043d11dc8a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.388157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.427787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba061c1d-b88a-4ee7-8ac3-89043d11dc8a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.465837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5d81d-89b1-4f7e-9ff4-728fec8316fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.466444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d81d-89b1-4f7e-9ff4-728fec8316fd None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.466775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d81d-89b1-4f7e-9ff4-728fec8316fd None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.467223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5d81d-89b1-4f7e-9ff4-728fec8316fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.467956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5d81d-89b1-4f7e-9ff4-728fec8316fd None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=f38e6571879e43c3a43f7378fb4a0ce1, user.id=a551c780c77945cebb37fee2d66a72aa) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.468259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5d81d-89b1-4f7e-9ff4-728fec8316fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.469801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d5d81d-89b1-4f7e-9ff4-728fec8316fd None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.476420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 532/1059] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/role_assignments?user.id=a551c780c77945cebb37fee2d66a72aa&scope.project.id=f38e6571879e43c3a43f7378fb4a0ce1&role.id=b698800ceec3460a8519ea627650da0f => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.894159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f279d9-d758-405a-a38f-4a2dbb01efa8 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.894521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f279d9-d758-405a-a38f-4a2dbb01efa8 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.894707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f279d9-d758-405a-a38f-4a2dbb01efa8 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.896238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 528/1060] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:09.902058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba721eb-e3c4-4867-b4fb-a78ae5a6cb7f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.902805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba721eb-e3c4-4867-b4fb-a78ae5a6cb7f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.902805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba721eb-e3c4-4867-b4fb-a78ae5a6cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.921923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-8ba721eb-e3c4-4867-b4fb-a78ae5a6cb7f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:09.923143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8ba721eb-e3c4-4867-b4fb-a78ae5a6cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:10.003616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 533/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:10.264488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92386c61-623f-4825-9955-d24729a11170 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.265007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92386c61-623f-4825-9955-d24729a11170 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.265007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92386c61-623f-4825-9955-d24729a11170 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.266522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 529/1062] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:10.272166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba721eb-e3c4-4867-b4fb-a78ae5a6cb7f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.272722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b4f4-4135-43a2-8d71-9c3905820b34 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.273018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b4f4-4135-43a2-8d71-9c3905820b34 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.273298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97b4f4-4135-43a2-8d71-9c3905820b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.293540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-9c97b4f4-4135-43a2-8d71-9c3905820b34 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:10.294794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9c97b4f4-4135-43a2-8d71-9c3905820b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:10.351946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.376853 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 530/1063] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:10.399432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba721eb-e3c4-4867-b4fb-a78ae5a6cb7f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.435248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7683e8d-81e0-493f-bfca-09d4462e0a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.435867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683e8d-81e0-493f-bfca-09d4462e0a66 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.436165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683e8d-81e0-493f-bfca-09d4462e0a66 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.436425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7683e8d-81e0-493f-bfca-09d4462e0a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.509337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7683e8d-81e0-493f-bfca-09d4462e0a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.510828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7683e8d-81e0-493f-bfca-09d4462e0a66 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.586301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.625672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 534/1064] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:10.630112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97b4f4-4135-43a2-8d71-9c3905820b34 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.691991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7683e8d-81e0-493f-bfca-09d4462e0a66 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.747430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97b4f4-4135-43a2-8d71-9c3905820b34 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.759115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.789076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.789763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.790043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.790316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.813874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7683e8d-81e0-493f-bfca-09d4462e0a66 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.816588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.818238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:10.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:10.837313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:10.838309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:10.838309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:10.838309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:10.838309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 531/1065] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:10.843600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.868159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047194-fa75-4b76-b1ab-26d0cd052f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.868979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047194-fa75-4b76-b1ab-26d0cd052f12 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.869267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047194-fa75-4b76-b1ab-26d0cd052f12 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.869540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5047194-fa75-4b76-b1ab-26d0cd052f12 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.880653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5047194-fa75-4b76-b1ab-26d0cd052f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-c5047194-fa75-4b76-b1ab-26d0cd052f12 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:10.893822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:10.894881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 535/1066] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:10.895768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047194-fa75-4b76-b1ab-26d0cd052f12 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.930996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.989181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a0fa8-7ce5-4f40-af5d-65f649eccc57 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.026158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5047194-fa75-4b76-b1ab-26d0cd052f12 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.033680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.034308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.034593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.034869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.035534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.035836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.037829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.051821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 532/1067] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.072924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.073571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.073884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.074159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.074856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.075157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.076566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.086738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 536/1068] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:11.092554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.121776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.222423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecadb23-8ac3-49a1-8af1-b829e089eaf1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.262547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.263186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.263460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.263734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.275182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.276665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:11.294309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:11.295337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:11.296255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:11.296255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:11.296255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:11.296255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 533/1069] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:11.301725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.405965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.419996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d59661-2ed7-4129-a5db-446deab822a7 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.456370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db40310-bd57-452d-b2ee-cb0977e11f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.456991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40310-bd57-452d-b2ee-cb0977e11f25 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.457270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40310-bd57-452d-b2ee-cb0977e11f25 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.457542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db40310-bd57-452d-b2ee-cb0977e11f25 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.458232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db40310-bd57-452d-b2ee-cb0977e11f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.458535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db40310-bd57-452d-b2ee-cb0977e11f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.460456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db40310-bd57-452d-b2ee-cb0977e11f25 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.474149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 537/1070] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.480371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.564419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.594046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593e032-e7d7-4653-8ffe-b714fb558805 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.628685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.629321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.629614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.629904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.638579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.640014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.643787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48e39-8206-428f-9490-4cdc6d2a1314 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.644074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48e39-8206-428f-9490-4cdc6d2a1314 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.644352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd48e39-8206-428f-9490-4cdc6d2a1314 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.645726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 538/1071] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:11.651967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57764e-1ec3-4d93-a488-fe9192007242 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.652330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57764e-1ec3-4d93-a488-fe9192007242 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.652742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57764e-1ec3-4d93-a488-fe9192007242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:11.657053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:11.658204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:11.659169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 534/1072] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:11.664189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.671949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-1e57764e-1ec3-4d93-a488-fe9192007242 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:11.673166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1e57764e-1ec3-4d93-a488-fe9192007242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:11.748117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 539/1073] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:11.788158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e44e9b-2543-44b1-995a-fdea2cbcd452 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.823672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.824276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.824557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.824830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.825558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.825896 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.827334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.829160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.847916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 535/1074] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.853965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57764e-1ec3-4d93-a488-fe9192007242 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.942687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.965288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e57764e-1ec3-4d93-a488-fe9192007242 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.000020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.000706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.000981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.001255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.015399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.016910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.018942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:12.037106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:12.038235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:12.039149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:12.039149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:12.039149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:12.039149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 540/1075] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:12.044205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.154032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7ebce-9d16-466a-9755-4351920321fa None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.160129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74986999-eddf-44a8-b064-ff79e061f70d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.189268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.192176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a28f4-deaf-4e8c-8668-a44df81ca34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.193172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a28f4-deaf-4e8c-8668-a44df81ca34a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.193285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a28f4-deaf-4e8c-8668-a44df81ca34a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.193692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a28f4-deaf-4e8c-8668-a44df81ca34a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.198132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.198781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.199054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.199320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.199967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:12.200264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.201629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.207290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a28f4-deaf-4e8c-8668-a44df81ca34a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-028a28f4-deaf-4e8c-8668-a44df81ca34a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:12.212272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:12.213539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:12.214446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 536/1076] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:12.215569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 541/1077] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:12.221032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a28f4-deaf-4e8c-8668-a44df81ca34a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.221908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.281177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.332748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3289e-1a60-4d20-998b-4b6c0998b1e4 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.335896 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a28f4-deaf-4e8c-8668-a44df81ca34a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.369559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.370340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.370620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.370890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.371413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.371567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:12.371855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.372059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.372346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.372623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.373233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.381816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.382736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 542/1078] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:12.383331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.388501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:12.395109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:12.396717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:12.397959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 537/1079] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:12.402711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.459253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.509288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174f9b8-06bc-4da5-87df-9089949fc371 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.533161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd9683-a082-436f-9ed1-ac520c737015 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.547651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.549647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6388eb-77d2-468f-b4ed-d62965415c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.550283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6388eb-77d2-468f-b4ed-d62965415c48 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.550568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6388eb-77d2-468f-b4ed-d62965415c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.550832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6388eb-77d2-468f-b4ed-d62965415c48 None admin] PATH_INFO: `/v3/users/a551c780c77945cebb37fee2d66a72aa` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.551982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6388eb-77d2-468f-b4ed-d62965415c48 None admin] RBAC: Authorizing `identity:get_user(user_id=a551c780c77945cebb37fee2d66a72aa)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.554164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6388eb-77d2-468f-b4ed-d62965415c48 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.560440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 543/1080] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/users/a551c780c77945cebb37fee2d66a72aa => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:12.561506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6388eb-77d2-468f-b4ed-d62965415c48 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.571186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.571801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.572081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.572349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.573046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:12.573350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.576338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.588879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 538/1081] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:12.594579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.668421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6388eb-77d2-468f-b4ed-d62965415c48 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.691099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.703126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78c2a4-1af9-4c9e-9892-475a67cf6a5e None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.711338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b95d2-89d9-46f9-b1a7-073035757c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.711895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b95d2-89d9-46f9-b1a7-073035757c1c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.712170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b95d2-89d9-46f9-b1a7-073035757c1c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.712437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b95d2-89d9-46f9-b1a7-073035757c1c None admin] PATH_INFO: `/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.713557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b95d2-89d9-46f9-b1a7-073035757c1c None admin] RBAC: Authorizing `identity:get_project(project_id=f38e6571879e43c3a43f7378fb4a0ce1)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.715009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b95d2-89d9-46f9-b1a7-073035757c1c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.716659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 544/1082] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:12.722541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b95d2-89d9-46f9-b1a7-073035757c1c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.740664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab5668-3b60-40f8-97fb-5590922d0584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.741341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab5668-3b60-40f8-97fb-5590922d0584 None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.741648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab5668-3b60-40f8-97fb-5590922d0584 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.741922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab5668-3b60-40f8-97fb-5590922d0584 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/2981bb0558d34329a061a5f08149bba3/roles/719de673a5ee4eb8a7e528f62ec5603d` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.744255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab5668-3b60-40f8-97fb-5590922d0584 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=2981bb0558d34329a061a5f08149bba3, role_id=719de673a5ee4eb8a7e528f62ec5603d)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.745789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab5668-3b60-40f8-97fb-5590922d0584 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.760009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 539/1083] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:12 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/2981bb0558d34329a061a5f08149bba3/roles/719de673a5ee4eb8a7e528f62ec5603d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:12.854765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b95d2-89d9-46f9-b1a7-073035757c1c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.898119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c45d0-a594-44a0-8f87-ac99a4bb2b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.898824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c45d0-a594-44a0-8f87-ac99a4bb2b08 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.899189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c45d0-a594-44a0-8f87-ac99a4bb2b08 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.899465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c45d0-a594-44a0-8f87-ac99a4bb2b08 None admin] PATH_INFO: `/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1/users/a551c780c77945cebb37fee2d66a72aa/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.901802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c45d0-a594-44a0-8f87-ac99a4bb2b08 None admin] RBAC: Authorizing `identity:create_grant(project_id=f38e6571879e43c3a43f7378fb4a0ce1, user_id=a551c780c77945cebb37fee2d66a72aa, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.904177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c45d0-a594-44a0-8f87-ac99a4bb2b08 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.921419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 545/1084] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:12 2022] PUT /identity/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1/users/a551c780c77945cebb37fee2d66a72aa/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:14.945583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f56b87-d783-4d42-8f59-86794d1f8a25 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.946279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f56b87-d783-4d42-8f59-86794d1f8a25 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.946279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f56b87-d783-4d42-8f59-86794d1f8a25 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.947524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 540/1085] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:14.953162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d442e3-b682-47f2-a336-f734161b8d32 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.953658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d442e3-b682-47f2-a336-f734161b8d32 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.953770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d442e3-b682-47f2-a336-f734161b8d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.972061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-36d442e3-b682-47f2-a336-f734161b8d32 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:14.973264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-36d442e3-b682-47f2-a336-f734161b8d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:15.032702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.059974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 546/1086] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:15.119184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439ed1a-276f-417f-b2f2-fa43d75023bc None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.119593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439ed1a-276f-417f-b2f2-fa43d75023bc None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.119942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439ed1a-276f-417f-b2f2-fa43d75023bc None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.121344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 541/1087] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:15.126752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891824a9-0cb3-44ff-8feb-4670c85aec7f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.127040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891824a9-0cb3-44ff-8feb-4670c85aec7f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.127323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891824a9-0cb3-44ff-8feb-4670c85aec7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.147189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-891824a9-0cb3-44ff-8feb-4670c85aec7f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:15.148394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-891824a9-0cb3-44ff-8feb-4670c85aec7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:15.223852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 547/1088] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:15.327790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439ed1a-276f-417f-b2f2-fa43d75023bc None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.359208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.443513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439ed1a-276f-417f-b2f2-fa43d75023bc None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.493695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d737c-92a0-4aa5-a65e-e2cd4f5e5f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.494608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d737c-92a0-4aa5-a65e-e2cd4f5e5f2d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.495051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d737c-92a0-4aa5-a65e-e2cd4f5e5f2d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.495531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d737c-92a0-4aa5-a65e-e2cd4f5e5f2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.504641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891824a9-0cb3-44ff-8feb-4670c85aec7f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.508192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d737c-92a0-4aa5-a65e-e2cd4f5e5f2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-0a6d737c-92a0-4aa5-a65e-e2cd4f5e5f2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:15.513222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:15.514227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:15.515071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 542/1089] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:15.523162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d737c-92a0-4aa5-a65e-e2cd4f5e5f2d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.610932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891824a9-0cb3-44ff-8feb-4670c85aec7f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.621693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.622285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.631865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d737c-92a0-4aa5-a65e-e2cd4f5e5f2d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.650665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc8373-4ccd-4ea7-aefd-61139412091e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.651374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc8373-4ccd-4ea7-aefd-61139412091e None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.651708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc8373-4ccd-4ea7-aefd-61139412091e None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.652049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc8373-4ccd-4ea7-aefd-61139412091e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.663182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc8373-4ccd-4ea7-aefd-61139412091e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:15.667562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-53fc8373-4ccd-4ea7-aefd-61139412091e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:15.668040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:15.668970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:15.670022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:15.670915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.670915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.670915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.671146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 548/1090] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:15.673876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:15.674335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.675285 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc8373-4ccd-4ea7-aefd-61139412091e None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.675730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.688629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 543/1091] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:15.694730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.787298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc8373-4ccd-4ea7-aefd-61139412091e None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.801876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420ec4a-4486-4be2-bf66-4f79c0c4d827 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.825536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.826351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.826962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.827235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.827902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:15.828200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.829584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.839123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 549/1092] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:15.841110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.842214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.842669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.843159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.844824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.856303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.857809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:15.873196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:15.879237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:15.880192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:15.880192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:15.880192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:15.880192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 544/1093] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:15.880192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.881213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.889843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.967369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0badd0-bd2a-478d-8a54-5757d308e4d5 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.990813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c6849f-bff5-43cc-b74d-4d18d3d22fd0 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.004359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a836-2025-42b8-ba4b-6724864b4384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.004993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a836-2025-42b8-ba4b-6724864b4384 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.005265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a836-2025-42b8-ba4b-6724864b4384 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.005838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a836-2025-42b8-ba4b-6724864b4384 None admin] PATH_INFO: `/v3/users/a551c780c77945cebb37fee2d66a72aa` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.007022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0a836-2025-42b8-ba4b-6724864b4384 None admin] RBAC: Authorizing `identity:get_user(user_id=a551c780c77945cebb37fee2d66a72aa)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.008444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0a836-2025-42b8-ba4b-6724864b4384 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.010447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 550/1094] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/users/a551c780c77945cebb37fee2d66a72aa => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:16.016605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a836-2025-42b8-ba4b-6724864b4384 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.033134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.033783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.034071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.034396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.035040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:16.035341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.036954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.050688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 545/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:16.056596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.127083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.127718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.131416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a836-2025-42b8-ba4b-6724864b4384 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.166510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4499a1-264f-4e0d-9fec-bc4f8197af47 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.168686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fa56f-da68-4cf0-ac62-ab68d907f8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.169301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa56f-da68-4cf0-ac62-ab68d907f8f7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.169573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa56f-da68-4cf0-ac62-ab68d907f8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.169880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686fa56f-da68-4cf0-ac62-ab68d907f8f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.207700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.208341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.209190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.209477 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.217157 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.218658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:16.231023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:16.232312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:16.233504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 546/1096] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:16.238034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.251608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fa56f-da68-4cf0-ac62-ab68d907f8f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.253074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686fa56f-da68-4cf0-ac62-ab68d907f8f7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.352191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fed56-79ea-4fe7-aa27-69e7ead7e6c8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.360788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.370269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 551/1097] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:16.371663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fa56f-da68-4cf0-ac62-ab68d907f8f7 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.383385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.396840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.397807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.398270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.398722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.399962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:16.400408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.402936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.431182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 547/1098] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:16.436920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.499903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686fa56f-da68-4cf0-ac62-ab68d907f8f7 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.541699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f08422-2e51-4e73-856d-191415b057ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.542473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f08422-2e51-4e73-856d-191415b057ab None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.542749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f08422-2e51-4e73-856d-191415b057ab None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.543012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f08422-2e51-4e73-856d-191415b057ab None admin] PATH_INFO: `/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.544133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f08422-2e51-4e73-856d-191415b057ab None admin] RBAC: Authorizing `identity:get_project(project_id=f38e6571879e43c3a43f7378fb4a0ce1)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.545129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ce995-4a60-49bd-a307-6b98c6294ef1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.545551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f08422-2e51-4e73-856d-191415b057ab None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.547235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 552/1099] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:16.553842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f08422-2e51-4e73-856d-191415b057ab None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.585039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104993c5-8667-4d6d-8b1d-fa492fed2256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.585707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104993c5-8667-4d6d-8b1d-fa492fed2256 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.586029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104993c5-8667-4d6d-8b1d-fa492fed2256 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.586219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104993c5-8667-4d6d-8b1d-fa492fed2256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.623256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.643089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.659929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104993c5-8667-4d6d-8b1d-fa492fed2256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.661409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104993c5-8667-4d6d-8b1d-fa492fed2256 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.664186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f08422-2e51-4e73-856d-191415b057ab None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.699531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.700128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.700399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.700736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.701382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=f38e6571879e43c3a43f7378fb4a0ce1, user.id=a551c780c77945cebb37fee2d66a72aa) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:16.701718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.703146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.709116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 553/1100] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/role_assignments?user.id=a551c780c77945cebb37fee2d66a72aa&scope.project.id=f38e6571879e43c3a43f7378fb4a0ce1&role.id=b698800ceec3460a8519ea627650da0f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.780594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 548/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:37:16.787121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.900505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021fe188-0a95-4c7d-ae43-17373bfc9cdc None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.940717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.949057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.949754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.950021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.950293 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.961985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.963446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:16.979260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:16.980969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:16.982307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:16.982307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:16.982307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:16.982307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 554/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:16.988153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104993c5-8667-4d6d-8b1d-fa492fed2256 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.082079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.099650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104993c5-8667-4d6d-8b1d-fa492fed2256 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.142194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.142817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.143105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.143376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.144036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:17.144344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.145752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.158030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 549/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:17.164276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.273879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b43f6-b2ac-41a1-89aa-9cd6c1ef9c56 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.314074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.314723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.315007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.315284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.322926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.324396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:17.336865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:17.337911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:17.338908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 555/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:17.343982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.463019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89413f98-4dde-4a15-a044-586f3b270d95 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.506147 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba520e-2e2c-4c06-9066-ccafbcd273e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.506826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba520e-2e2c-4c06-9066-ccafbcd273e1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.507238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba520e-2e2c-4c06-9066-ccafbcd273e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.507680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba520e-2e2c-4c06-9066-ccafbcd273e1 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.508941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba520e-2e2c-4c06-9066-ccafbcd273e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:17.509493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba520e-2e2c-4c06-9066-ccafbcd273e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.512068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba520e-2e2c-4c06-9066-ccafbcd273e1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.525556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 550/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:17.532398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.577707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.655580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56f917-5661-4147-9b0a-f8fe35a35bd0 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.696760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acabf11-cdb0-4ca7-8d26-35e9daf246d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.697735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acabf11-cdb0-4ca7-8d26-35e9daf246d5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.697735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acabf11-cdb0-4ca7-8d26-35e9daf246d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.698004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acabf11-cdb0-4ca7-8d26-35e9daf246d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.698656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acabf11-cdb0-4ca7-8d26-35e9daf246d5 None admin] RBAC: Adding query filter params (role.id=719de673a5ee4eb8a7e528f62ec5603d, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=2981bb0558d34329a061a5f08149bba3) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:17.699058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acabf11-cdb0-4ca7-8d26-35e9daf246d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.700824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acabf11-cdb0-4ca7-8d26-35e9daf246d5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.712499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 556/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/role_assignments?user.id=2981bb0558d34329a061a5f08149bba3&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=719de673a5ee4eb8a7e528f62ec5603d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:18.986476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ff3b7-2ab9-45f3-814d-01215d5d76ef None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.987023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ff3b7-2ab9-45f3-814d-01215d5d76ef None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.987023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ff3b7-2ab9-45f3-814d-01215d5d76ef None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.988379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 551/1107] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:18.994012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525067c-9b7f-4aa5-8f7e-d0c530adeb40 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.994551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525067c-9b7f-4aa5-8f7e-d0c530adeb40 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.994551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525067c-9b7f-4aa5-8f7e-d0c530adeb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.012775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4525067c-9b7f-4aa5-8f7e-d0c530adeb40 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:19.014050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4525067c-9b7f-4aa5-8f7e-d0c530adeb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:19.084938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 557/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:19.351757 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ff3b7-2ab9-45f3-814d-01215d5d76ef None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.355562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.459078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ff3b7-2ab9-45f3-814d-01215d5d76ef None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.495839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669a647-0313-4979-9ef6-52ccab638a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.496472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a647-0313-4979-9ef6-52ccab638a77 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.496752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a647-0313-4979-9ef6-52ccab638a77 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.497026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a647-0313-4979-9ef6-52ccab638a77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.498167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669a647-0313-4979-9ef6-52ccab638a77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.499614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669a647-0313-4979-9ef6-52ccab638a77 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.501193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 552/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.507730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525067c-9b7f-4aa5-8f7e-d0c530adeb40 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.541441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.606303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525067c-9b7f-4aa5-8f7e-d0c530adeb40 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.641262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2e939b-aa29-439e-a783-5b2b8b0af116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.641908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e939b-aa29-439e-a783-5b2b8b0af116 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.642183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e939b-aa29-439e-a783-5b2b8b0af116 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.642453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2e939b-aa29-439e-a783-5b2b8b0af116 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.643108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e939b-aa29-439e-a783-5b2b8b0af116 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.644542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2e939b-aa29-439e-a783-5b2b8b0af116 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.666048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 558/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:19 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:21.899108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e049-2a1f-4887-9c79-a164d50deea7 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.899708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e049-2a1f-4887-9c79-a164d50deea7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.899708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4e049-2a1f-4887-9c79-a164d50deea7 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.901010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 553/1111] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:21.906894 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72734e-3958-42b9-a0de-ae0048322292 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.907416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72734e-3958-42b9-a0de-ae0048322292 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.907416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72734e-3958-42b9-a0de-ae0048322292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.926080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-be72734e-3958-42b9-a0de-ae0048322292 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:21.927849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-be72734e-3958-42b9-a0de-ae0048322292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:21.998128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 559/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:22.265346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4e049-2a1f-4887-9c79-a164d50deea7 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.339317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.372152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4e049-2a1f-4887-9c79-a164d50deea7 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.407752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c163da-44e8-40cc-96ff-84a608c46d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.408386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c163da-44e8-40cc-96ff-84a608c46d47 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.408665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c163da-44e8-40cc-96ff-84a608c46d47 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.408939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c163da-44e8-40cc-96ff-84a608c46d47 None admin] PATH_INFO: `/v3/roles/90097e8ce1394daba58b200de808ad59` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.410359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c163da-44e8-40cc-96ff-84a608c46d47 None admin] RBAC: Authorizing `identity:get_role(role_id=90097e8ce1394daba58b200de808ad59)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.411777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c163da-44e8-40cc-96ff-84a608c46d47 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.413122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 554/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/roles/90097e8ce1394daba58b200de808ad59 => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:22.419027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72734e-3958-42b9-a0de-ae0048322292 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.443721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.522430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72734e-3958-42b9-a0de-ae0048322292 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.557102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e3866-54d4-4325-a1bf-74d682b330a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.557772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3866-54d4-4325-a1bf-74d682b330a0 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.558090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3866-54d4-4325-a1bf-74d682b330a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.558363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e3866-54d4-4325-a1bf-74d682b330a0 None admin] PATH_INFO: `/v3/users/272d1fbc2365483884fee3ec0771e35e` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.573237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e3866-54d4-4325-a1bf-74d682b330a0 None admin] RBAC: Authorizing `identity:get_user(user_id=272d1fbc2365483884fee3ec0771e35e)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.574734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11e3866-54d4-4325-a1bf-74d682b330a0 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.576461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 560/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/users/272d1fbc2365483884fee3ec0771e35e => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:22.582353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c163da-44e8-40cc-96ff-84a608c46d47 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.683040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c163da-44e8-40cc-96ff-84a608c46d47 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.719055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ef6a3-3bb7-4ded-9e7e-4d592777d905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.719665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ef6a3-3bb7-4ded-9e7e-4d592777d905 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.719944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ef6a3-3bb7-4ded-9e7e-4d592777d905 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.720213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ef6a3-3bb7-4ded-9e7e-4d592777d905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.731602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.791006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ef6a3-3bb7-4ded-9e7e-4d592777d905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.792384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ef6a3-3bb7-4ded-9e7e-4d592777d905 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.897797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 555/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:22.903776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e3866-54d4-4325-a1bf-74d682b330a0 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:23.011617 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:23.033029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e3866-54d4-4325-a1bf-74d682b330a0 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.068343 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb024c02-d035-4df4-b206-0af3afdb8a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.069157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb024c02-d035-4df4-b206-0af3afdb8a23 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.069435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb024c02-d035-4df4-b206-0af3afdb8a23 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.069739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb024c02-d035-4df4-b206-0af3afdb8a23 None admin] PATH_INFO: `/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.070889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb024c02-d035-4df4-b206-0af3afdb8a23 None admin] RBAC: Authorizing `identity:get_project(project_id=f38e6571879e43c3a43f7378fb4a0ce1)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.072324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb024c02-d035-4df4-b206-0af3afdb8a23 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.074009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 561/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:23.080209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ef6a3-3bb7-4ded-9e7e-4d592777d905 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:23.122478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:23.189850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ef6a3-3bb7-4ded-9e7e-4d592777d905 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.225435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c4c8c-3650-4118-9689-b3781e454e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.226063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c4c8c-3650-4118-9689-b3781e454e81 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.226340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c4c8c-3650-4118-9689-b3781e454e81 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.226612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c4c8c-3650-4118-9689-b3781e454e81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.227330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c4c8c-3650-4118-9689-b3781e454e81 None admin] RBAC: Adding query filter params (role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=f38e6571879e43c3a43f7378fb4a0ce1, user.id=272d1fbc2365483884fee3ec0771e35e) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:23.227631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c4c8c-3650-4118-9689-b3781e454e81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.229074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c4c8c-3650-4118-9689-b3781e454e81 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.234717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 556/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:23 2022] GET /identity/v3/role_assignments?user.id=272d1fbc2365483884fee3ec0771e35e&scope.project.id=f38e6571879e43c3a43f7378fb4a0ce1&role.id=90097e8ce1394daba58b200de808ad59 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:25.396585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f649c-05bc-433a-b535-dcd1ca3e5d36 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.400645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f649c-05bc-433a-b535-dcd1ca3e5d36 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.400645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f649c-05bc-433a-b535-dcd1ca3e5d36 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.400645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 562/1118] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:25.403734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c133030-6653-4664-8add-4a295fdd1a17 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.404158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c133030-6653-4664-8add-4a295fdd1a17 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.404283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c133030-6653-4664-8add-4a295fdd1a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.423233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8c133030-6653-4664-8add-4a295fdd1a17 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:25.424344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8c133030-6653-4664-8add-4a295fdd1a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:25.492215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 557/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:25.762457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f649c-05bc-433a-b535-dcd1ca3e5d36 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.868716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f649c-05bc-433a-b535-dcd1ca3e5d36 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.902976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9735b-49d3-4fe1-bf56-0d2575152183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.903611 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9735b-49d3-4fe1-bf56-0d2575152183 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.903878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9735b-49d3-4fe1-bf56-0d2575152183 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.904141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9735b-49d3-4fe1-bf56-0d2575152183 None admin] PATH_INFO: `/v3/roles/90097e8ce1394daba58b200de808ad59` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.905563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9735b-49d3-4fe1-bf56-0d2575152183 None admin] RBAC: Authorizing `identity:get_role(role_id=90097e8ce1394daba58b200de808ad59)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.906963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9735b-49d3-4fe1-bf56-0d2575152183 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.908301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 563/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/roles/90097e8ce1394daba58b200de808ad59 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:25.914562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c133030-6653-4664-8add-4a295fdd1a17 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.947788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.028680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c133030-6653-4664-8add-4a295fdd1a17 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.069849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5a694-0bfb-4e44-94f1-92fcbbbfa2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.070653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5a694-0bfb-4e44-94f1-92fcbbbfa2e1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.070943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5a694-0bfb-4e44-94f1-92fcbbbfa2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.071219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5a694-0bfb-4e44-94f1-92fcbbbfa2e1 None admin] PATH_INFO: `/v3/users/272d1fbc2365483884fee3ec0771e35e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.072544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5a694-0bfb-4e44-94f1-92fcbbbfa2e1 None admin] RBAC: Authorizing `identity:get_user(user_id=272d1fbc2365483884fee3ec0771e35e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.074452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5a694-0bfb-4e44-94f1-92fcbbbfa2e1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.076326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 558/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/users/272d1fbc2365483884fee3ec0771e35e => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:26.082180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9735b-49d3-4fe1-bf56-0d2575152183 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.100746 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.194838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9735b-49d3-4fe1-bf56-0d2575152183 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.228125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda374-a09f-4deb-8eb8-d0c48747ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.228750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda374-a09f-4deb-8eb8-d0c48747ce3a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.229026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda374-a09f-4deb-8eb8-d0c48747ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.229303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda374-a09f-4deb-8eb8-d0c48747ce3a None admin] PATH_INFO: `/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.230461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cda374-a09f-4deb-8eb8-d0c48747ce3a None admin] RBAC: Authorizing `identity:get_project(project_id=f38e6571879e43c3a43f7378fb4a0ce1)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.231895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cda374-a09f-4deb-8eb8-d0c48747ce3a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.233537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 564/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:26 2022] GET /identity/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:26.240167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5a694-0bfb-4e44-94f1-92fcbbbfa2e1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.340543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5a694-0bfb-4e44-94f1-92fcbbbfa2e1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.342060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.376437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7607fbf-9094-44d8-b0af-ed796b2fae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.377100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7607fbf-9094-44d8-b0af-ed796b2fae42 None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.377379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7607fbf-9094-44d8-b0af-ed796b2fae42 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.377675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7607fbf-9094-44d8-b0af-ed796b2fae42 None admin] PATH_INFO: `/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1/users/272d1fbc2365483884fee3ec0771e35e/roles/90097e8ce1394daba58b200de808ad59` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.380413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7607fbf-9094-44d8-b0af-ed796b2fae42 None admin] RBAC: Authorizing `identity:create_grant(project_id=f38e6571879e43c3a43f7378fb4a0ce1, user_id=272d1fbc2365483884fee3ec0771e35e, role_id=90097e8ce1394daba58b200de808ad59)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.382018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7607fbf-9094-44d8-b0af-ed796b2fae42 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.395672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 559/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:26 2022] PUT /identity/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1/users/272d1fbc2365483884fee3ec0771e35e/roles/90097e8ce1394daba58b200de808ad59 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:28.559141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95263474-b5dc-43cd-b7e1-a6b87096d164 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.559716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95263474-b5dc-43cd-b7e1-a6b87096d164 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.560174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95263474-b5dc-43cd-b7e1-a6b87096d164 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.562340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 565/1124] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:28.568211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fce469-6437-465e-9f37-2b5941036b0f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.568938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fce469-6437-465e-9f37-2b5941036b0f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.569061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fce469-6437-465e-9f37-2b5941036b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.587850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b5fce469-6437-465e-9f37-2b5941036b0f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:28.589059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b5fce469-6437-465e-9f37-2b5941036b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:28.662495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 560/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:28.934276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95263474-b5dc-43cd-b7e1-a6b87096d164 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.027391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.044204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95263474-b5dc-43cd-b7e1-a6b87096d164 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.079522 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd30cc5-0556-48d1-a8c7-dda38e9a7bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.080154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd30cc5-0556-48d1-a8c7-dda38e9a7bfd None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.080434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd30cc5-0556-48d1-a8c7-dda38e9a7bfd None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.080702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd30cc5-0556-48d1-a8c7-dda38e9a7bfd None admin] PATH_INFO: `/v3/roles/90097e8ce1394daba58b200de808ad59` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.082119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd30cc5-0556-48d1-a8c7-dda38e9a7bfd None admin] RBAC: Authorizing `identity:get_role(role_id=90097e8ce1394daba58b200de808ad59)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.084163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd30cc5-0556-48d1-a8c7-dda38e9a7bfd None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.085494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 566/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/roles/90097e8ce1394daba58b200de808ad59 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.091405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fce469-6437-465e-9f37-2b5941036b0f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.184140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.194826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fce469-6437-465e-9f37-2b5941036b0f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.230650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ba12-e141-4f39-a6bc-2c7adfd13ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.231286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ba12-e141-4f39-a6bc-2c7adfd13ca1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.231566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ba12-e141-4f39-a6bc-2c7adfd13ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.231839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470ba12-e141-4f39-a6bc-2c7adfd13ca1 None admin] PATH_INFO: `/v3/users/272d1fbc2365483884fee3ec0771e35e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.232986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470ba12-e141-4f39-a6bc-2c7adfd13ca1 None admin] RBAC: Authorizing `identity:get_user(user_id=272d1fbc2365483884fee3ec0771e35e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.234454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470ba12-e141-4f39-a6bc-2c7adfd13ca1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.236068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 561/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/users/272d1fbc2365483884fee3ec0771e35e => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.245151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd30cc5-0556-48d1-a8c7-dda38e9a7bfd None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.351471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd30cc5-0556-48d1-a8c7-dda38e9a7bfd None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.386568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8d1f-1be2-457e-bcb3-970be79d7319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.387187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8d1f-1be2-457e-bcb3-970be79d7319 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.387460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8d1f-1be2-457e-bcb3-970be79d7319 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.387734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8d1f-1be2-457e-bcb3-970be79d7319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.409302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.456826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a8d1f-1be2-457e-bcb3-970be79d7319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.458218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a8d1f-1be2-457e-bcb3-970be79d7319 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.556931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 567/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:29.562796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ba12-e141-4f39-a6bc-2c7adfd13ca1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.671270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470ba12-e141-4f39-a6bc-2c7adfd13ca1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.706945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1f787-d3be-4e7c-9497-a249d5669faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.707643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f787-d3be-4e7c-9497-a249d5669faa None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.708025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f787-d3be-4e7c-9497-a249d5669faa None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.708403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1f787-d3be-4e7c-9497-a249d5669faa None admin] PATH_INFO: `/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.709675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1f787-d3be-4e7c-9497-a249d5669faa None admin] RBAC: Authorizing `identity:get_project(project_id=f38e6571879e43c3a43f7378fb4a0ce1)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.711129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1f787-d3be-4e7c-9497-a249d5669faa None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.712809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 562/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/projects/f38e6571879e43c3a43f7378fb4a0ce1 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.719185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8d1f-1be2-457e-bcb3-970be79d7319 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.772429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.828655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8d1f-1be2-457e-bcb3-970be79d7319 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.863540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667f5f1-18d1-47ee-9a1c-b8abad522a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.864700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f5f1-18d1-47ee-9a1c-b8abad522a55 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.864970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f5f1-18d1-47ee-9a1c-b8abad522a55 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.865232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f5f1-18d1-47ee-9a1c-b8abad522a55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.865977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667f5f1-18d1-47ee-9a1c-b8abad522a55 None admin] RBAC: Adding query filter params (role.id=90097e8ce1394daba58b200de808ad59, scope.project.id=f38e6571879e43c3a43f7378fb4a0ce1, user.id=272d1fbc2365483884fee3ec0771e35e) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.866281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667f5f1-18d1-47ee-9a1c-b8abad522a55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.867700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667f5f1-18d1-47ee-9a1c-b8abad522a55 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.873634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 568/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/role_assignments?user.id=272d1fbc2365483884fee3ec0771e35e&scope.project.id=f38e6571879e43c3a43f7378fb4a0ce1&role.id=90097e8ce1394daba58b200de808ad59 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:32.075187 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96233b2-b46f-4f67-aad7-61ca7e68acc6 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.075763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96233b2-b46f-4f67-aad7-61ca7e68acc6 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.075763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96233b2-b46f-4f67-aad7-61ca7e68acc6 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.077053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 563/1131] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:32.082563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee4699-5550-40f2-bde1-1d571b28a391 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.083302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee4699-5550-40f2-bde1-1d571b28a391 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.083302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee4699-5550-40f2-bde1-1d571b28a391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.103447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bdee4699-5550-40f2-bde1-1d571b28a391 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:32.104641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bdee4699-5550-40f2-bde1-1d571b28a391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:32.172733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 569/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:32.443336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96233b2-b46f-4f67-aad7-61ca7e68acc6 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.474437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.551823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96233b2-b46f-4f67-aad7-61ca7e68acc6 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.602306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b8028-38ee-422e-b4b7-579014d6434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.602941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8028-38ee-422e-b4b7-579014d6434b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.603226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8028-38ee-422e-b4b7-579014d6434b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.603499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b8028-38ee-422e-b4b7-579014d6434b None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.605002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b8028-38ee-422e-b4b7-579014d6434b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.606923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b8028-38ee-422e-b4b7-579014d6434b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.608514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 564/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:32.614428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee4699-5550-40f2-bde1-1d571b28a391 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.664934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.718809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee4699-5550-40f2-bde1-1d571b28a391 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.754013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59083fba-b62a-4e0a-abd0-1569faad9e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.755112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59083fba-b62a-4e0a-abd0-1569faad9e86 None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.755384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59083fba-b62a-4e0a-abd0-1569faad9e86 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.755647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59083fba-b62a-4e0a-abd0-1569faad9e86 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.756302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59083fba-b62a-4e0a-abd0-1569faad9e86 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.757773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59083fba-b62a-4e0a-abd0-1569faad9e86 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.788201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 570/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:32 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:35.035503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922767e-3015-4135-8a31-796b7cb19d88 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.036095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922767e-3015-4135-8a31-796b7cb19d88 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.036209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922767e-3015-4135-8a31-796b7cb19d88 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.037500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 565/1135] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:35.043221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea4b7-017a-429d-a066-592ae2621630 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.043726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea4b7-017a-429d-a066-592ae2621630 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.043828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea4b7-017a-429d-a066-592ae2621630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.065041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-064ea4b7-017a-429d-a066-592ae2621630 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:35.066283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-064ea4b7-017a-429d-a066-592ae2621630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:35.136056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 571/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:35.411000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922767e-3015-4135-8a31-796b7cb19d88 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.511326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.518689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922767e-3015-4135-8a31-796b7cb19d88 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.557700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f14c5-947f-413f-82aa-f0cb562a8c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.558334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f14c5-947f-413f-82aa-f0cb562a8c5f None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.558615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f14c5-947f-413f-82aa-f0cb562a8c5f None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.558887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f14c5-947f-413f-82aa-f0cb562a8c5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.559998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f14c5-947f-413f-82aa-f0cb562a8c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.561434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f14c5-947f-413f-82aa-f0cb562a8c5f None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.563046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 566/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:35.569889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ea4b7-017a-429d-a066-592ae2621630 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.596793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.669487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ea4b7-017a-429d-a066-592ae2621630 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.704039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30a51a-5855-4469-b0ff-44dd9b89086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.704646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a51a-5855-4469-b0ff-44dd9b89086c None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.704917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a51a-5855-4469-b0ff-44dd9b89086c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.705182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30a51a-5855-4469-b0ff-44dd9b89086c None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.705866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30a51a-5855-4469-b0ff-44dd9b89086c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.707315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30a51a-5855-4469-b0ff-44dd9b89086c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.731322 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 572/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:35 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:37.950082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b874-eaf2-4064-8c78-0064fc17b823 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.950653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b874-eaf2-4064-8c78-0064fc17b823 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.950653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b874-eaf2-4064-8c78-0064fc17b823 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.951960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 567/1139] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:37.958164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f0b92-071f-4545-a2e3-c7cc112e1a60 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.958716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f0b92-071f-4545-a2e3-c7cc112e1a60 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.958716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f0b92-071f-4545-a2e3-c7cc112e1a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.977411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e89f0b92-071f-4545-a2e3-c7cc112e1a60 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:37.978659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e89f0b92-071f-4545-a2e3-c7cc112e1a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:38.049380 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 573/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:38.329143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76b874-eaf2-4064-8c78-0064fc17b823 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.441138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76b874-eaf2-4064-8c78-0064fc17b823 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.477209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632ad75-6c02-4996-9f4c-d6296ba8684a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.477868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ad75-6c02-4996-9f4c-d6296ba8684a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.478148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ad75-6c02-4996-9f4c-d6296ba8684a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.478420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632ad75-6c02-4996-9f4c-d6296ba8684a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.489153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632ad75-6c02-4996-9f4c-d6296ba8684a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-a632ad75-6c02-4996-9f4c-d6296ba8684a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:38.494072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:38.495311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:38.496566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 568/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:38.501424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f0b92-071f-4545-a2e3-c7cc112e1a60 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.518109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.606597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f0b92-071f-4545-a2e3-c7cc112e1a60 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.642737 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.643345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.643625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.643900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.644568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.644905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.646318 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.655221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 574/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.661804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632ad75-6c02-4996-9f4c-d6296ba8684a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.689492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.771088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632ad75-6c02-4996-9f4c-d6296ba8684a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.808000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfbed0-f803-4faa-949e-91c30a859f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.808638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfbed0-f803-4faa-949e-91c30a859f3a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.808918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfbed0-f803-4faa-949e-91c30a859f3a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.809188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfbed0-f803-4faa-949e-91c30a859f3a None admin] PATH_INFO: `/v3/users/5227f3e15da9418599fec065c2785711` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.823717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfbed0-f803-4faa-949e-91c30a859f3a None admin] RBAC: Authorizing `identity:get_user(user_id=5227f3e15da9418599fec065c2785711)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.825167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfbed0-f803-4faa-949e-91c30a859f3a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.826888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 569/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/users/5227f3e15da9418599fec065c2785711 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.832798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.910838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.934942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81731c11-46a6-4a58-a195-56fa37ed9807 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.969693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3357725-cad8-4e04-8fdb-7b70a946366f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.970412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3357725-cad8-4e04-8fdb-7b70a946366f None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.970575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3357725-cad8-4e04-8fdb-7b70a946366f None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.970848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3357725-cad8-4e04-8fdb-7b70a946366f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.043650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3357725-cad8-4e04-8fdb-7b70a946366f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.045269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3357725-cad8-4e04-8fdb-7b70a946366f None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.124759 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.148631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 575/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:39.154846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfbed0-f803-4faa-949e-91c30a859f3a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.245226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.262469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfbed0-f803-4faa-949e-91c30a859f3a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.299164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c965f75-e465-4a15-b666-cd67cdeda3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.299803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c965f75-e465-4a15-b666-cd67cdeda3c9 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.300093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c965f75-e465-4a15-b666-cd67cdeda3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.300365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c965f75-e465-4a15-b666-cd67cdeda3c9 None admin] PATH_INFO: `/v3/projects/d42e8fae3ed8455296c74855690bbea9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.301479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c965f75-e465-4a15-b666-cd67cdeda3c9 None admin] RBAC: Authorizing `identity:get_project(project_id=d42e8fae3ed8455296c74855690bbea9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.302961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c965f75-e465-4a15-b666-cd67cdeda3c9 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.304617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 570/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/projects/d42e8fae3ed8455296c74855690bbea9 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.311350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3357725-cad8-4e04-8fdb-7b70a946366f None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.411156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3357725-cad8-4e04-8fdb-7b70a946366f None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.444554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d72a7-2c64-4613-9717-e6e50c1054d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.445151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d72a7-2c64-4613-9717-e6e50c1054d8 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.445424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d72a7-2c64-4613-9717-e6e50c1054d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.445717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d72a7-2c64-4613-9717-e6e50c1054d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.446435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d72a7-2c64-4613-9717-e6e50c1054d8 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=d42e8fae3ed8455296c74855690bbea9, user.id=5227f3e15da9418599fec065c2785711) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.446738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d72a7-2c64-4613-9717-e6e50c1054d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.448161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d72a7-2c64-4613-9717-e6e50c1054d8 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.453946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 576/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/role_assignments?user.id=5227f3e15da9418599fec065c2785711&scope.project.id=d42e8fae3ed8455296c74855690bbea9&role.id=b698800ceec3460a8519ea627650da0f => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:41.611527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21217be-c988-48ab-968d-7216518fd567 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.612217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21217be-c988-48ab-968d-7216518fd567 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.612217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21217be-c988-48ab-968d-7216518fd567 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.613402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 571/1147] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:41.621010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cf2fc-a1ea-4981-91bd-b0ef5c925c60 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.621538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cf2fc-a1ea-4981-91bd-b0ef5c925c60 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.621538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cf2fc-a1ea-4981-91bd-b0ef5c925c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.640337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-455cf2fc-a1ea-4981-91bd-b0ef5c925c60 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:41.643505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-455cf2fc-a1ea-4981-91bd-b0ef5c925c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:41.678019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:41.714697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 577/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:41.985880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21217be-c988-48ab-968d-7216518fd567 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.092099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21217be-c988-48ab-968d-7216518fd567 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.131140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58344b5b-7aa3-483b-a26c-3ec21ddd1878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.131776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58344b5b-7aa3-483b-a26c-3ec21ddd1878 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.132058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58344b5b-7aa3-483b-a26c-3ec21ddd1878 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.132324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58344b5b-7aa3-483b-a26c-3ec21ddd1878 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.143097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58344b5b-7aa3-483b-a26c-3ec21ddd1878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-58344b5b-7aa3-483b-a26c-3ec21ddd1878 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:42.148104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:42.149290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:42.150196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 572/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:42.155475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cf2fc-a1ea-4981-91bd-b0ef5c925c60 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.257388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cf2fc-a1ea-4981-91bd-b0ef5c925c60 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.295578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.296191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.296472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.296742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.297411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:42.297740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.299102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.308214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 578/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:42.314644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58344b5b-7aa3-483b-a26c-3ec21ddd1878 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.328427 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.417950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58344b5b-7aa3-483b-a26c-3ec21ddd1878 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.452784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850c1f3-5727-417e-ae22-3fda6b5aeb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.453422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c1f3-5727-417e-ae22-3fda6b5aeb82 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.453730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c1f3-5727-417e-ae22-3fda6b5aeb82 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.454004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c1f3-5727-417e-ae22-3fda6b5aeb82 None admin] PATH_INFO: `/v3/users/5227f3e15da9418599fec065c2785711` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.455144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850c1f3-5727-417e-ae22-3fda6b5aeb82 None admin] RBAC: Authorizing `identity:get_user(user_id=5227f3e15da9418599fec065c2785711)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.456597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850c1f3-5727-417e-ae22-3fda6b5aeb82 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.458299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 573/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/users/5227f3e15da9418599fec065c2785711 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:42.464218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.499163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.569270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b1dce-d22c-4356-858b-c19a596e02da None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.603813 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d9042-a73e-4686-a2fa-a31c6e56d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.604444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d9042-a73e-4686-a2fa-a31c6e56d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.604717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d9042-a73e-4686-a2fa-a31c6e56d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.604981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d9042-a73e-4686-a2fa-a31c6e56d3a7 None admin] PATH_INFO: `/v3/projects/d42e8fae3ed8455296c74855690bbea9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.606136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d9042-a73e-4686-a2fa-a31c6e56d3a7 None admin] RBAC: Authorizing `identity:get_project(project_id=d42e8fae3ed8455296c74855690bbea9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.607564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d9042-a73e-4686-a2fa-a31c6e56d3a7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.609250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 579/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/projects/d42e8fae3ed8455296c74855690bbea9 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:42.616127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850c1f3-5727-417e-ae22-3fda6b5aeb82 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.701216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.724844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850c1f3-5727-417e-ae22-3fda6b5aeb82 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.759513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122a9bf-dcb4-46e9-b9cb-cb0d9e06043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.760181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122a9bf-dcb4-46e9-b9cb-cb0d9e06043a None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.760464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122a9bf-dcb4-46e9-b9cb-cb0d9e06043a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.760893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122a9bf-dcb4-46e9-b9cb-cb0d9e06043a None admin] PATH_INFO: `/v3/projects/d42e8fae3ed8455296c74855690bbea9/users/5227f3e15da9418599fec065c2785711/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.762970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122a9bf-dcb4-46e9-b9cb-cb0d9e06043a None admin] RBAC: Authorizing `identity:create_grant(project_id=d42e8fae3ed8455296c74855690bbea9, user_id=5227f3e15da9418599fec065c2785711, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.764426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122a9bf-dcb4-46e9-b9cb-cb0d9e06043a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.780100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 574/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:42 2022] PUT /identity/v3/projects/d42e8fae3ed8455296c74855690bbea9/users/5227f3e15da9418599fec065c2785711/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:44.942991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadcbbb-4c04-453f-8af4-34145a5dd308 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.943863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadcbbb-4c04-453f-8af4-34145a5dd308 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.943863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadcbbb-4c04-453f-8af4-34145a5dd308 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.945136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 580/1154] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:44.950830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78581676-639f-4492-aceb-f76d0a5d1662 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.951317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78581676-639f-4492-aceb-f76d0a5d1662 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.951414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78581676-639f-4492-aceb-f76d0a5d1662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.970018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-78581676-639f-4492-aceb-f76d0a5d1662 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:44.971221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-78581676-639f-4492-aceb-f76d0a5d1662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:45.041099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 575/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:45.315307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadcbbb-4c04-453f-8af4-34145a5dd308 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.415522 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.419553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadcbbb-4c04-453f-8af4-34145a5dd308 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.453921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322fb0c-b6ae-4cd2-8ee6-fa51318656ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.454557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322fb0c-b6ae-4cd2-8ee6-fa51318656ed None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.454834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322fb0c-b6ae-4cd2-8ee6-fa51318656ed None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.455105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3322fb0c-b6ae-4cd2-8ee6-fa51318656ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.465838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3322fb0c-b6ae-4cd2-8ee6-fa51318656ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-3322fb0c-b6ae-4cd2-8ee6-fa51318656ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:45.470679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.471940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:37:45.473557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 581/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.478499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78581676-639f-4492-aceb-f76d0a5d1662 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.545381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.580550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78581676-639f-4492-aceb-f76d0a5d1662 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.615139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.615749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.616028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.616300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.616974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.617270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.618690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.627337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 576/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:45.633411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322fb0c-b6ae-4cd2-8ee6-fa51318656ed None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.737777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3322fb0c-b6ae-4cd2-8ee6-fa51318656ed None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.771941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c811-2725-424b-9f25-c34e71e281ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.772568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c811-2725-424b-9f25-c34e71e281ad None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.772846 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c811-2725-424b-9f25-c34e71e281ad None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.773115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86c811-2725-424b-9f25-c34e71e281ad None admin] PATH_INFO: `/v3/users/5227f3e15da9418599fec065c2785711` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.774252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c811-2725-424b-9f25-c34e71e281ad None admin] RBAC: Authorizing `identity:get_user(user_id=5227f3e15da9418599fec065c2785711)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.775688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a86c811-2725-424b-9f25-c34e71e281ad None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.777294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 582/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/users/5227f3e15da9418599fec065c2785711 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:45.783199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.884936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a6483-57e3-4f27-8ef1-7148f413614a None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.915480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.921700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410afc60-6584-45bd-af92-ace8d97bb880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.922310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410afc60-6584-45bd-af92-ace8d97bb880 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.922588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410afc60-6584-45bd-af92-ace8d97bb880 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.922857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410afc60-6584-45bd-af92-ace8d97bb880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.991056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410afc60-6584-45bd-af92-ace8d97bb880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.992877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410afc60-6584-45bd-af92-ace8d97bb880 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.102285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 577/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:46.108640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c811-2725-424b-9f25-c34e71e281ad None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.136888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.216991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86c811-2725-424b-9f25-c34e71e281ad None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.254372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b489e8-0faf-4e55-acb4-ccf7c0f78e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.255012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b489e8-0faf-4e55-acb4-ccf7c0f78e48 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.255286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b489e8-0faf-4e55-acb4-ccf7c0f78e48 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.255556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b489e8-0faf-4e55-acb4-ccf7c0f78e48 None admin] PATH_INFO: `/v3/projects/d42e8fae3ed8455296c74855690bbea9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.256672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b489e8-0faf-4e55-acb4-ccf7c0f78e48 None admin] RBAC: Authorizing `identity:get_project(project_id=d42e8fae3ed8455296c74855690bbea9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.258143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b489e8-0faf-4e55-acb4-ccf7c0f78e48 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.259790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 583/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/projects/d42e8fae3ed8455296c74855690bbea9 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.266101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410afc60-6584-45bd-af92-ace8d97bb880 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.310025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.420488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410afc60-6584-45bd-af92-ace8d97bb880 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.456894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c36d94-2cd4-44f5-a376-4b8fc3a79a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.457501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c36d94-2cd4-44f5-a376-4b8fc3a79a74 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.457810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c36d94-2cd4-44f5-a376-4b8fc3a79a74 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.458082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c36d94-2cd4-44f5-a376-4b8fc3a79a74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.458801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c36d94-2cd4-44f5-a376-4b8fc3a79a74 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=d42e8fae3ed8455296c74855690bbea9, user.id=5227f3e15da9418599fec065c2785711) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.459106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c36d94-2cd4-44f5-a376-4b8fc3a79a74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.460553 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c36d94-2cd4-44f5-a376-4b8fc3a79a74 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.467278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 578/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/role_assignments?user.id=5227f3e15da9418599fec065c2785711&scope.project.id=d42e8fae3ed8455296c74855690bbea9&role.id=b698800ceec3460a8519ea627650da0f => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:48.679769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d70ae-a841-4aed-8168-664bf2cbaaa5 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.680336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d70ae-a841-4aed-8168-664bf2cbaaa5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.680336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d70ae-a841-4aed-8168-664bf2cbaaa5 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.681687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 584/1162] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:48.687082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6407-c221-4ed0-b3b6-9dbcc3e2c022 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.687562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6407-c221-4ed0-b3b6-9dbcc3e2c022 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.687656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a6407-c221-4ed0-b3b6-9dbcc3e2c022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.706213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c30a6407-c221-4ed0-b3b6-9dbcc3e2c022 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:48.707984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c30a6407-c221-4ed0-b3b6-9dbcc3e2c022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:48.779509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 579/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:49.062237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d70ae-a841-4aed-8168-664bf2cbaaa5 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.164876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.172078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d70ae-a841-4aed-8168-664bf2cbaaa5 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.207324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583202ff-fd7d-4544-8d12-3a8a157cdb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.207938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583202ff-fd7d-4544-8d12-3a8a157cdb15 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.208219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583202ff-fd7d-4544-8d12-3a8a157cdb15 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.208491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583202ff-fd7d-4544-8d12-3a8a157cdb15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.281707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583202ff-fd7d-4544-8d12-3a8a157cdb15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.283093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583202ff-fd7d-4544-8d12-3a8a157cdb15 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.389680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.402214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 585/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:49.408302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a6407-c221-4ed0-b3b6-9dbcc3e2c022 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.438970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.528153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a6407-c221-4ed0-b3b6-9dbcc3e2c022 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.561863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47214f-7aea-420d-9020-a4fd42edbbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.562411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47214f-7aea-420d-9020-a4fd42edbbf3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.562687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47214f-7aea-420d-9020-a4fd42edbbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.562955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47214f-7aea-420d-9020-a4fd42edbbf3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.570665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47214f-7aea-420d-9020-a4fd42edbbf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.572112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47214f-7aea-420d-9020-a4fd42edbbf3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-2c47214f-7aea-420d-9020-a4fd42edbbf3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:49.584214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:49.589767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:37:49.590820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 580/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:49.591668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583202ff-fd7d-4544-8d12-3a8a157cdb15 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.700614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583202ff-fd7d-4544-8d12-3a8a157cdb15 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.735172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ad6be-64c5-4e79-ab6c-85f41777a7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.735782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ad6be-64c5-4e79-ab6c-85f41777a7f4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.736065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ad6be-64c5-4e79-ab6c-85f41777a7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.736339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ad6be-64c5-4e79-ab6c-85f41777a7f4 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.736979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ad6be-64c5-4e79-ab6c-85f41777a7f4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.737282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ad6be-64c5-4e79-ab6c-85f41777a7f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.738709 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ad6be-64c5-4e79-ab6c-85f41777a7f4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.746042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 586/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:51.906675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c288ef-57e4-4573-9d0f-0578152cb05d None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.907299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c288ef-57e4-4573-9d0f-0578152cb05d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.907299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c288ef-57e4-4573-9d0f-0578152cb05d None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.908542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 581/1167] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:51.914663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cb846-f93e-47b7-ba47-d3ce4c70fe65 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.915159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cb846-f93e-47b7-ba47-d3ce4c70fe65 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.915271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26cb846-f93e-47b7-ba47-d3ce4c70fe65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.933963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f26cb846-f93e-47b7-ba47-d3ce4c70fe65 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:51.935168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f26cb846-f93e-47b7-ba47-d3ce4c70fe65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:51.992359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.004506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 587/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:52.273697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c288ef-57e4-4573-9d0f-0578152cb05d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.362514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.379282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c288ef-57e4-4573-9d0f-0578152cb05d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.413665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bf00a-31fc-4eb0-9562-b4a9d300100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.414271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bf00a-31fc-4eb0-9562-b4a9d300100a None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.414548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bf00a-31fc-4eb0-9562-b4a9d300100a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.414817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bf00a-31fc-4eb0-9562-b4a9d300100a None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.415488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bf00a-31fc-4eb0-9562-b4a9d300100a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.416856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bf00a-31fc-4eb0-9562-b4a9d300100a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.434363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 582/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Tue Sep 13 19:37:52 2022] POST /identity/v3/domains => generated 267 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:54.613908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a14a-e96c-46b3-9809-ef71da2795e2 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:54.614454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a14a-e96c-46b3-9809-ef71da2795e2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:54.614454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a14a-e96c-46b3-9809-ef71da2795e2 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:54.615797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 588/1170] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:54.621195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6c25c-1383-4384-b55f-21e1e5f7245e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:54.621688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6c25c-1383-4384-b55f-21e1e5f7245e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:54.621785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6c25c-1383-4384-b55f-21e1e5f7245e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:54.648500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1fc6c25c-1383-4384-b55f-21e1e5f7245e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:54.649735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1fc6c25c-1383-4384-b55f-21e1e5f7245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:54.717447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 583/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:54.994348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a14a-e96c-46b3-9809-ef71da2795e2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.094700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a14a-e96c-46b3-9809-ef71da2795e2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.127868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998ce88-f175-4353-9b48-4b5189fcad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.128498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ce88-f175-4353-9b48-4b5189fcad97 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.128774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ce88-f175-4353-9b48-4b5189fcad97 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.129049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ce88-f175-4353-9b48-4b5189fcad97 None admin] PATH_INFO: `/v3/domains/13ec779ed05f47c2a8baa9bd3e8ed805` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.139220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998ce88-f175-4353-9b48-4b5189fcad97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=13ec779ed05f47c2a8baa9bd3e8ed805)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:55.140663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998ce88-f175-4353-9b48-4b5189fcad97 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:55.142306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 589/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Sep 13 19:37:54 2022] GET /identity/v3/domains/13ec779ed05f47c2a8baa9bd3e8ed805 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:55.149515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6c25c-1383-4384-b55f-21e1e5f7245e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.272714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:55.305154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc6c25c-1383-4384-b55f-21e1e5f7245e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.341713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510093e-fcc0-4012-b3dd-9b5b4d34ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.342319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510093e-fcc0-4012-b3dd-9b5b4d34ee2a None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.342596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510093e-fcc0-4012-b3dd-9b5b4d34ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.342862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510093e-fcc0-4012-b3dd-9b5b4d34ee2a None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.343508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510093e-fcc0-4012-b3dd-9b5b4d34ee2a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:55.345056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510093e-fcc0-4012-b3dd-9b5b4d34ee2a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:55.371413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 584/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:55 2022] POST /identity/v3/projects => generated 361 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:57.653598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe32208-f8b1-49be-8086-af240dd6b81b None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.654174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe32208-f8b1-49be-8086-af240dd6b81b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.654174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe32208-f8b1-49be-8086-af240dd6b81b None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.655514 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 590/1174] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:57.661122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe426-e537-4187-8f21-cd349c765bd4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.661577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe426-e537-4187-8f21-cd349c765bd4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.661695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe426-e537-4187-8f21-cd349c765bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.680137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-deefe426-e537-4187-8f21-cd349c765bd4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:57.681317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-deefe426-e537-4187-8f21-cd349c765bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:57.752579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 585/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:58.022433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe32208-f8b1-49be-8086-af240dd6b81b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.076969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.129881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe32208-f8b1-49be-8086-af240dd6b81b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.164719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a664cfd-8cf8-4986-900d-58d6f99c260c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.165354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664cfd-8cf8-4986-900d-58d6f99c260c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.165722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664cfd-8cf8-4986-900d-58d6f99c260c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.166001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a664cfd-8cf8-4986-900d-58d6f99c260c None admin] PATH_INFO: `/v3/domains/13ec779ed05f47c2a8baa9bd3e8ed805` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.167126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a664cfd-8cf8-4986-900d-58d6f99c260c None admin] RBAC: Authorizing `identity:get_domain(domain_id=13ec779ed05f47c2a8baa9bd3e8ed805)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.168561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a664cfd-8cf8-4986-900d-58d6f99c260c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.170179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 591/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/domains/13ec779ed05f47c2a8baa9bd3e8ed805 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:58.176394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefe426-e537-4187-8f21-cd349c765bd4 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.230999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.273844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefe426-e537-4187-8f21-cd349c765bd4 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.313241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e700051-48d8-41a6-9fd1-5589b77a2642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.313912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e700051-48d8-41a6-9fd1-5589b77a2642 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.314148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e700051-48d8-41a6-9fd1-5589b77a2642 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.314414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e700051-48d8-41a6-9fd1-5589b77a2642 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.315073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e700051-48d8-41a6-9fd1-5589b77a2642 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.316535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e700051-48d8-41a6-9fd1-5589b77a2642 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.341978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 586/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:58 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:00.538704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0886b-e82b-4080-b6b7-20226e29d404 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.539272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0886b-e82b-4080-b6b7-20226e29d404 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.539272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0886b-e82b-4080-b6b7-20226e29d404 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.540577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 592/1178] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:00.545799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acbb6c-9335-4623-aede-2290605ab7fd None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.546237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acbb6c-9335-4623-aede-2290605ab7fd None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.546355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acbb6c-9335-4623-aede-2290605ab7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.564558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-64acbb6c-9335-4623-aede-2290605ab7fd None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:00.566321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-64acbb6c-9335-4623-aede-2290605ab7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:00.633046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 587/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:00.903008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0886b-e82b-4080-b6b7-20226e29d404 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.008101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0886b-e82b-4080-b6b7-20226e29d404 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.025984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.043931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bb951-cad6-46bd-b791-8db6bfc997fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.044559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bb951-cad6-46bd-b791-8db6bfc997fd None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.044836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bb951-cad6-46bd-b791-8db6bfc997fd None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.045107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bb951-cad6-46bd-b791-8db6bfc997fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.056073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bb951-cad6-46bd-b791-8db6bfc997fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-3f9bb951-cad6-46bd-b791-8db6bfc997fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:01.060895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:01.062019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:38:01.062931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 593/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:01.068372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acbb6c-9335-4623-aede-2290605ab7fd None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.111019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.171552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acbb6c-9335-4623-aede-2290605ab7fd None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.204151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.204748 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.205024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.205294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.206490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.206791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.208153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.215889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 588/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:01.222738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bb951-cad6-46bd-b791-8db6bfc997fd None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.320397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bb951-cad6-46bd-b791-8db6bfc997fd None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.354291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c69aa-6d66-476d-a7fd-3271c93bee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.354918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c69aa-6d66-476d-a7fd-3271c93bee3a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.355192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c69aa-6d66-476d-a7fd-3271c93bee3a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.355459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c69aa-6d66-476d-a7fd-3271c93bee3a None admin] PATH_INFO: `/v3/users/abcdcdf12cab4a799725169cb2ff1f38` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.368145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c69aa-6d66-476d-a7fd-3271c93bee3a None admin] RBAC: Authorizing `identity:get_user(user_id=abcdcdf12cab4a799725169cb2ff1f38)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.369572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c69aa-6d66-476d-a7fd-3271c93bee3a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.371194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 594/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/users/abcdcdf12cab4a799725169cb2ff1f38 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:01.376992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.473998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfdcff7-406e-454f-a414-94f01780f634 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.476128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.506572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc545bd6-f283-4306-bbfe-4d4eea819143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.507158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc545bd6-f283-4306-bbfe-4d4eea819143 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.507437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc545bd6-f283-4306-bbfe-4d4eea819143 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.507708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc545bd6-f283-4306-bbfe-4d4eea819143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.571929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc545bd6-f283-4306-bbfe-4d4eea819143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.573369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc545bd6-f283-4306-bbfe-4d4eea819143 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.672342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 589/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:01.679094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c69aa-6d66-476d-a7fd-3271c93bee3a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.716752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.782748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c69aa-6d66-476d-a7fd-3271c93bee3a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.816797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b015432c-f31e-4a2f-8979-d855c108cbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.817418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015432c-f31e-4a2f-8979-d855c108cbc4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.817718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015432c-f31e-4a2f-8979-d855c108cbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.818242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015432c-f31e-4a2f-8979-d855c108cbc4 None admin] PATH_INFO: `/v3/projects/255393b93a944833a7293e8f8d43bfc2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.819355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015432c-f31e-4a2f-8979-d855c108cbc4 None admin] RBAC: Authorizing `identity:get_project(project_id=255393b93a944833a7293e8f8d43bfc2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.820776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b015432c-f31e-4a2f-8979-d855c108cbc4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.822454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 595/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/projects/255393b93a944833a7293e8f8d43bfc2 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:01.828775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc545bd6-f283-4306-bbfe-4d4eea819143 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.834118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.936043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc545bd6-f283-4306-bbfe-4d4eea819143 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.970177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8541302-3fad-4b83-8f1d-3ff7afcfad10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.970776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8541302-3fad-4b83-8f1d-3ff7afcfad10 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.971053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8541302-3fad-4b83-8f1d-3ff7afcfad10 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.971323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8541302-3fad-4b83-8f1d-3ff7afcfad10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.972063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8541302-3fad-4b83-8f1d-3ff7afcfad10 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=255393b93a944833a7293e8f8d43bfc2, user.id=abcdcdf12cab4a799725169cb2ff1f38) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.972367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8541302-3fad-4b83-8f1d-3ff7afcfad10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.973843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8541302-3fad-4b83-8f1d-3ff7afcfad10 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.979135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 590/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/role_assignments?user.id=abcdcdf12cab4a799725169cb2ff1f38&scope.project.id=255393b93a944833a7293e8f8d43bfc2&role.id=b698800ceec3460a8519ea627650da0f => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:04.113209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751db1c3-afba-4d06-864c-ba48e961a768 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.113781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751db1c3-afba-4d06-864c-ba48e961a768 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.114142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751db1c3-afba-4d06-864c-ba48e961a768 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.116385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 596/1186] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:38:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:04.121849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e880f8-784b-48e6-8526-699703ac4378 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.122303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e880f8-784b-48e6-8526-699703ac4378 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.122494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e880f8-784b-48e6-8526-699703ac4378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.141632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e2e880f8-784b-48e6-8526-699703ac4378 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:04.143028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e2e880f8-784b-48e6-8526-699703ac4378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:04.218568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.226739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 591/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:04.507459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751db1c3-afba-4d06-864c-ba48e961a768 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.610850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751db1c3-afba-4d06-864c-ba48e961a768 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.613186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.645001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde4ac5-bbab-4cd8-b3b3-83c0d3b62886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.645650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4ac5-bbab-4cd8-b3b3-83c0d3b62886 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.645926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4ac5-bbab-4cd8-b3b3-83c0d3b62886 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.646195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4ac5-bbab-4cd8-b3b3-83c0d3b62886 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.656029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde4ac5-bbab-4cd8-b3b3-83c0d3b62886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-bdde4ac5-bbab-4cd8-b3b3-83c0d3b62886 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:04.660916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:04.662050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:38:04.663236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 597/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:04.668103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e880f8-784b-48e6-8526-699703ac4378 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.764497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e880f8-784b-48e6-8526-699703ac4378 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.797564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.798196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.798482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.798756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.799438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:04.799819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.801129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.809228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 592/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:04.815580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde4ac5-bbab-4cd8-b3b3-83c0d3b62886 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.930419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde4ac5-bbab-4cd8-b3b3-83c0d3b62886 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.986338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11224f-617c-49c3-8724-4ae09d285eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.987945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11224f-617c-49c3-8724-4ae09d285eff None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.988447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11224f-617c-49c3-8724-4ae09d285eff None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.988947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a11224f-617c-49c3-8724-4ae09d285eff None admin] PATH_INFO: `/v3/users/abcdcdf12cab4a799725169cb2ff1f38` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.990791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11224f-617c-49c3-8724-4ae09d285eff None admin] RBAC: Authorizing `identity:get_user(user_id=abcdcdf12cab4a799725169cb2ff1f38)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.993351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a11224f-617c-49c3-8724-4ae09d285eff None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.996331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 598/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/users/abcdcdf12cab4a799725169cb2ff1f38 => generated 311 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:05.002151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.062259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.104475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce911b-d7fa-45a2-b7e4-a54026a0ee55 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.138934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724aca9-3bf4-4c04-b470-b9cdcc320ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.139623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724aca9-3bf4-4c04-b470-b9cdcc320ce0 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.139836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724aca9-3bf4-4c04-b470-b9cdcc320ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.140112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724aca9-3bf4-4c04-b470-b9cdcc320ce0 None admin] PATH_INFO: `/v3/projects/255393b93a944833a7293e8f8d43bfc2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.141216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724aca9-3bf4-4c04-b470-b9cdcc320ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=255393b93a944833a7293e8f8d43bfc2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.142687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724aca9-3bf4-4c04-b470-b9cdcc320ce0 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.144405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 593/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/projects/255393b93a944833a7293e8f8d43bfc2 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:05.150454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11224f-617c-49c3-8724-4ae09d285eff None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.181162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.250272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a11224f-617c-49c3-8724-4ae09d285eff None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.284014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71217933-6b46-4afb-a60c-42917eed17ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.284681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71217933-6b46-4afb-a60c-42917eed17ec None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.284861 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71217933-6b46-4afb-a60c-42917eed17ec None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.285128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71217933-6b46-4afb-a60c-42917eed17ec None admin] PATH_INFO: `/v3/projects/255393b93a944833a7293e8f8d43bfc2/users/abcdcdf12cab4a799725169cb2ff1f38/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.287052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71217933-6b46-4afb-a60c-42917eed17ec None admin] RBAC: Authorizing `identity:create_grant(project_id=255393b93a944833a7293e8f8d43bfc2, user_id=abcdcdf12cab4a799725169cb2ff1f38, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.288479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71217933-6b46-4afb-a60c-42917eed17ec None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.302823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 599/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:05 2022] PUT /identity/v3/projects/255393b93a944833a7293e8f8d43bfc2/users/abcdcdf12cab4a799725169cb2ff1f38/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:07.442327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff70c-76a0-4a43-add3-c7a7a784a362 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.443170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff70c-76a0-4a43-add3-c7a7a784a362 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.443170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ff70c-76a0-4a43-add3-c7a7a784a362 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.444493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 594/1193] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:07.449951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d70211-c3a0-44c6-b43d-d48377acd9d6 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.450386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d70211-c3a0-44c6-b43d-d48377acd9d6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.450482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d70211-c3a0-44c6-b43d-d48377acd9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.468811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-60d70211-c3a0-44c6-b43d-d48377acd9d6 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:07.470037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-60d70211-c3a0-44c6-b43d-d48377acd9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:07.535852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 600/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:07.805021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ff70c-76a0-4a43-add3-c7a7a784a362 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.905772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ff70c-76a0-4a43-add3-c7a7a784a362 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.931845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.941810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c34bbf-1b36-43e1-a81c-83e295d07417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.942451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c34bbf-1b36-43e1-a81c-83e295d07417 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.942991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c34bbf-1b36-43e1-a81c-83e295d07417 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.943270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c34bbf-1b36-43e1-a81c-83e295d07417 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.954023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c34bbf-1b36-43e1-a81c-83e295d07417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-d4c34bbf-1b36-43e1-a81c-83e295d07417 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:07.959009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:07.960210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:38:07.961494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 595/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:07.966319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d70211-c3a0-44c6-b43d-d48377acd9d6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.989934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.066110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d70211-c3a0-44c6-b43d-d48377acd9d6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.099802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.100411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.100692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.100967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.101652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.101960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.103325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.111387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 601/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.117921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c34bbf-1b36-43e1-a81c-83e295d07417 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.219869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c34bbf-1b36-43e1-a81c-83e295d07417 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.253953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049422e2-8681-4e27-b0a0-9188bd8f6ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.254589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049422e2-8681-4e27-b0a0-9188bd8f6ac4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.254869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049422e2-8681-4e27-b0a0-9188bd8f6ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.255143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049422e2-8681-4e27-b0a0-9188bd8f6ac4 None admin] PATH_INFO: `/v3/users/abcdcdf12cab4a799725169cb2ff1f38` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.256280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049422e2-8681-4e27-b0a0-9188bd8f6ac4 None admin] RBAC: Authorizing `identity:get_user(user_id=abcdcdf12cab4a799725169cb2ff1f38)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.257755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049422e2-8681-4e27-b0a0-9188bd8f6ac4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.259356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 596/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/users/abcdcdf12cab4a799725169cb2ff1f38 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.265312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.351794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.368572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de6f35-f002-4c6a-82c6-6f4e95ada18c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.401258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e5569-dacb-40da-85ab-9005008bc6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.401892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e5569-dacb-40da-85ab-9005008bc6d6 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.402180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e5569-dacb-40da-85ab-9005008bc6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.402451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e5569-dacb-40da-85ab-9005008bc6d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.477489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e5569-dacb-40da-85ab-9005008bc6d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.479693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e5569-dacb-40da-85ab-9005008bc6d6 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.569449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.586237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 602/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:08.592304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049422e2-8681-4e27-b0a0-9188bd8f6ac4 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.649449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.709360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049422e2-8681-4e27-b0a0-9188bd8f6ac4 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.742242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4cd95-0f64-4317-989a-fd34458dadbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.742878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4cd95-0f64-4317-989a-fd34458dadbc None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.743159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4cd95-0f64-4317-989a-fd34458dadbc None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.743432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4cd95-0f64-4317-989a-fd34458dadbc None admin] PATH_INFO: `/v3/projects/255393b93a944833a7293e8f8d43bfc2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.744543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4cd95-0f64-4317-989a-fd34458dadbc None admin] RBAC: Authorizing `identity:get_project(project_id=255393b93a944833a7293e8f8d43bfc2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.746019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b4cd95-0f64-4317-989a-fd34458dadbc None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.747654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 597/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/projects/255393b93a944833a7293e8f8d43bfc2 => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.753800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e5569-dacb-40da-85ab-9005008bc6d6 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.850334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e5569-dacb-40da-85ab-9005008bc6d6 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.882151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c4852-0910-45f7-974b-145ed7c175b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.882747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c4852-0910-45f7-974b-145ed7c175b4 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.883021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c4852-0910-45f7-974b-145ed7c175b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.883294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c4852-0910-45f7-974b-145ed7c175b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.884006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c4852-0910-45f7-974b-145ed7c175b4 None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=255393b93a944833a7293e8f8d43bfc2, user.id=abcdcdf12cab4a799725169cb2ff1f38) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.884313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c4852-0910-45f7-974b-145ed7c175b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.885768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c4852-0910-45f7-974b-145ed7c175b4 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.891264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 603/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/role_assignments?user.id=abcdcdf12cab4a799725169cb2ff1f38&scope.project.id=255393b93a944833a7293e8f8d43bfc2&role.id=b698800ceec3460a8519ea627650da0f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:18.742472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=38 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:18.888770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=76 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:05.113086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149dfe8-f006-4b37-9392-63c2898ae407 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:05.113729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149dfe8-f006-4b37-9392-63c2898ae407 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:05.113729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149dfe8-f006-4b37-9392-63c2898ae407 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:05.115354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 598/1201] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:05.120970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca47946-da69-4e6c-ab18-c0d960e18371 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:05.121487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca47946-da69-4e6c-ab18-c0d960e18371 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:05.121577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca47946-da69-4e6c-ab18-c0d960e18371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:05.145842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fca47946-da69-4e6c-ab18-c0d960e18371 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:05.147076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fca47946-da69-4e6c-ab18-c0d960e18371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:05.231332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 604/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:05.544161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149dfe8-f006-4b37-9392-63c2898ae407 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:05.694609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149dfe8-f006-4b37-9392-63c2898ae407 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:05.740562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:05.741262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:05.741656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:05.741948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:05.755471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:05.757201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:05.777012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:05.780394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:05.788205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:05.788205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:05.788205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:05.788205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 599/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:05.788764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca47946-da69-4e6c-ab18-c0d960e18371 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:05.936738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca47946-da69-4e6c-ab18-c0d960e18371 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:05.973278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:05.980110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2b705-bec0-4e68-b9dd-f557699d1980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:05.980962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2b705-bec0-4e68-b9dd-f557699d1980 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:05.981331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2b705-bec0-4e68-b9dd-f557699d1980 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:05.981792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2b705-bec0-4e68-b9dd-f557699d1980 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:05.982614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2b705-bec0-4e68-b9dd-f557699d1980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:05.982964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2b705-bec0-4e68-b9dd-f557699d1980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:05.984523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2b705-bec0-4e68-b9dd-f557699d1980 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:06.001678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 605/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:06.007727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:06.098507 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:06.177432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339881aa-8c2c-4209-b65a-35d8782e7d28 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:06.238683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48472e78-720a-4d69-b8e8-5f31e820f52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:06.239532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472e78-720a-4d69-b8e8-5f31e820f52a None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:06.239852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472e78-720a-4d69-b8e8-5f31e820f52a None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:06.240336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48472e78-720a-4d69-b8e8-5f31e820f52a None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:06.241074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48472e78-720a-4d69-b8e8-5f31e820f52a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:06.242923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48472e78-720a-4d69-b8e8-5f31e820f52a None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:06.277337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 600/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Sep 13 19:39:06 2022] POST /identity/v3/users => generated 251 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:08.708672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3e92f-1533-42ee-848a-31053d2a2a5c None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:08.709255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3e92f-1533-42ee-848a-31053d2a2a5c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:08.709255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3e92f-1533-42ee-848a-31053d2a2a5c None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:08.710650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 606/1206] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:08.716496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f27ac-c346-4578-9160-4f6da8a385a8 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:08.718061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f27ac-c346-4578-9160-4f6da8a385a8 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:08.718061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f27ac-c346-4578-9160-4f6da8a385a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:08.738625 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-0f8f27ac-c346-4578-9160-4f6da8a385a8 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:08.739857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0f8f27ac-c346-4578-9160-4f6da8a385a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:08.830746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 601/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:09.135520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3e92f-1533-42ee-848a-31053d2a2a5c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:09.280881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3e92f-1533-42ee-848a-31053d2a2a5c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:09.317983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a2cd5-2fa1-4b1f-8851-ecfe031b23a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:09.318621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a2cd5-2fa1-4b1f-8851-ecfe031b23a6 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:09.318904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a2cd5-2fa1-4b1f-8851-ecfe031b23a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:09.319175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a2cd5-2fa1-4b1f-8851-ecfe031b23a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:09.330524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a2cd5-2fa1-4b1f-8851-ecfe031b23a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-5f1a2cd5-2fa1-4b1f-8851-ecfe031b23a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:09.335400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:09.338694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:09.339623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 607/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:39:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:09.342768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f27ac-c346-4578-9160-4f6da8a385a8 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:09.387559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:09.455559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f27ac-c346-4578-9160-4f6da8a385a8 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:09.500417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:09.501064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:09.501346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:09.501654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:09.502375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:09.502675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:09.504254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:09.523717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 602/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:39:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:09.524951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a2cd5-2fa1-4b1f-8851-ecfe031b23a6 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:09.592155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:09.660358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a2cd5-2fa1-4b1f-8851-ecfe031b23a6 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:09.711158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:09.711900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:09.712316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:09.712709 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:09.725939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:09.727672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:09.748421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:09.749870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:09.750966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:09.750966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:09.750966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:09.750966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 608/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:39:09.755882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:09.896206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:09.900014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a3196-671d-43f7-996f-6983f5d4ef80 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:09.937734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:09.938490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:09.938766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:09.939038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:09.939684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:09.940000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:09.941816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:09.961040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 603/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:09.961365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:10.088312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0df9c0-8318-419f-a5de-f5b9fda8d6c7 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:10.090024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:10.129973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:10.130615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:10.130895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:10.131164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:10.140902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:10.142367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:39:10.154721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:10.161058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:10.162417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 609/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:39:10.164953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:10.291294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4d875-09cf-4328-92d8-8879941e4212 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:10.335047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:10.335698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:10.335940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:10.336238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:10.336970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:10.337305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:10.338761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:10.359515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 604/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 13 19:39:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:10.365681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:10.526997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d6cb4-e376-4827-8da7-b73cf9cdb0fd None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:10.566661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e081dfc-7953-4d0c-8233-0b4980411552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:10.568675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e081dfc-7953-4d0c-8233-0b4980411552 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:10.568675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e081dfc-7953-4d0c-8233-0b4980411552 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:10.568675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e081dfc-7953-4d0c-8233-0b4980411552 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:10.604607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:10.654415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e081dfc-7953-4d0c-8233-0b4980411552 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:10.656297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e081dfc-7953-4d0c-8233-0b4980411552 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:10.767677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 610/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:10.773584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:10.794327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:10.901112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac580fd-3354-4794-87d3-174ef8dd7103 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:10.940339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:10.941237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:10.941521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:10.941823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:10.952766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:10.954383 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:10.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:10.972499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:10.973829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:10.973829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:10.973829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:10.977104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 605/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:10.978184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e081dfc-7953-4d0c-8233-0b4980411552 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:11.054291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:11.096078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e081dfc-7953-4d0c-8233-0b4980411552 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:11.133288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:11.134001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:11.134266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:11.135567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:11.135567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:11.135567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:11.136908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:11.159044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 611/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:11.160330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:11.260262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:11.301363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86a2e-f29f-4ad7-bc68-92a086b6e0c2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:11.338254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:11.338718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:11.338988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:11.339261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:11.347057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:11.348760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:39:11.362230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:11.363384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:11.364680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 606/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:11.369458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:11.480364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd5c62-6a2a-4c1c-a186-1fb35e40a477 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:11.507144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:11.520873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3b9d0-73e8-4ab9-ba49-8f18848a6522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:11.521492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3b9d0-73e8-4ab9-ba49-8f18848a6522 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:11.521790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3b9d0-73e8-4ab9-ba49-8f18848a6522 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:11.522057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3b9d0-73e8-4ab9-ba49-8f18848a6522 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:11.522774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3b9d0-73e8-4ab9-ba49-8f18848a6522 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:11.523084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3b9d0-73e8-4ab9-ba49-8f18848a6522 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:11.524500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3b9d0-73e8-4ab9-ba49-8f18848a6522 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:11.539767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 612/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:11.548085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:11.676724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ecc21-78cb-4b18-90fa-93adebd6e373 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:11.713823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:11.720999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4a04c-a826-4af0-888e-95d7d07e3750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:11.721631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4a04c-a826-4af0-888e-95d7d07e3750 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:11.721991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4a04c-a826-4af0-888e-95d7d07e3750 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:11.722185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4a04c-a826-4af0-888e-95d7d07e3750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:11.722905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4a04c-a826-4af0-888e-95d7d07e3750 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=8957fc12ab74474f890b9188ba632fab) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:11.723244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4a04c-a826-4af0-888e-95d7d07e3750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:11.724650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4a04c-a826-4af0-888e-95d7d07e3750 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:11.733483 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 607/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:11 2022] GET /identity/v3/role_assignments?user.id=8957fc12ab74474f890b9188ba632fab&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:13.905839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e82c8-9af4-47eb-ab35-586579ee6488 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:13.906455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e82c8-9af4-47eb-ab35-586579ee6488 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:13.906455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e82c8-9af4-47eb-ab35-586579ee6488 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:13.907714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 613/1220] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:13.913028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c52a5e-f40b-4974-9f2c-29f793ae5b2c None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:13.913492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c52a5e-f40b-4974-9f2c-29f793ae5b2c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:13.913584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c52a5e-f40b-4974-9f2c-29f793ae5b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:13.933717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-61c52a5e-f40b-4974-9f2c-29f793ae5b2c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:13.934916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-61c52a5e-f40b-4974-9f2c-29f793ae5b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:14.023720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 608/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:14.296930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e82c8-9af4-47eb-ab35-586579ee6488 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:14.439879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e82c8-9af4-47eb-ab35-586579ee6488 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:14.477443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7ab0d-e373-4ec8-a9fd-58bf02906172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:14.478126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7ab0d-e373-4ec8-a9fd-58bf02906172 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:14.478403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7ab0d-e373-4ec8-a9fd-58bf02906172 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:14.478676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7ab0d-e373-4ec8-a9fd-58bf02906172 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:14.490261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7ab0d-e373-4ec8-a9fd-58bf02906172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-f5a7ab0d-e373-4ec8-a9fd-58bf02906172 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:14.495159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:14.496671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:14.497747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 614/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:39:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:39:14.503507 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c52a5e-f40b-4974-9f2c-29f793ae5b2c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:14.638069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c52a5e-f40b-4974-9f2c-29f793ae5b2c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:14.664652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:14.677130 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:14.677845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:14.678174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:14.678445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:14.679113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] RBAC: Adding query filter params (name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:14.679414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:14.682173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:14.693052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 609/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:39:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:14.699397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7ab0d-e373-4ec8-a9fd-58bf02906172 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:14.748356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:14.811433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7ab0d-e373-4ec8-a9fd-58bf02906172 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:14.849229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:14.849907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:14.850204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:14.850461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:14.862005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:14.863542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:14.885359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:14.887162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:14.888139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:14.888139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:14.888139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:14.888139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 615/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:39:14.893916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:15.029300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373f2ab-f03d-4e44-9a0a-6ca06af335fa None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:15.068879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:15.069499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:15.069812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:15.070121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:15.070816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:15.071196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:15.072746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:15.091038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 610/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:15.092320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:15.190412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:15.197828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa3df-4394-4a31-802f-9c51536708b3 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:15.233618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:15.234280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:15.234555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:15.234824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] PATH_INFO: `/v3/users/placement` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:15.242400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:15.243834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:39:15.255172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:15.256520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:15.257692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 616/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:15.262455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:15.319423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:15.389596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aec80c-7b0b-48f6-a7c5-cc801d634116 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:15.429437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:15.430154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:15.430470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:15.430851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:15.431673 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:15.431953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:15.433354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:15.454473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 611/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 13 19:39:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:15.460716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:15.599298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c133c-c63b-48fd-90f8-1f38f4336eaf None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:15.642499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:15.643536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:15.644087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:15.644504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:15.659341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:15.660811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:15.677455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:15.684454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:15.687261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:15.687261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:15.687261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:15.687261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 617/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:15.687805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:15.807966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfefc73-99d9-44ef-8d0e-5936f84e3edd None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:15.810288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:15.849271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:15.850005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:15.850287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:15.850586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:15.851235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:15.851540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:15.852918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:15.870084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 612/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:15.875446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:15.886610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:15.991464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f336cd2-b7ba-41b3-b46e-9eb2a65ec515 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:16.035997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:16.036610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:16.036892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:16.037157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:16.045203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:16.047492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:39:16.059087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:16.060113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:16.061073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 618/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:39:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:16.067080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:16.183387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a2c91-8451-4e09-a050-f1437089c0af None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:16.223117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247d89a-6faa-4233-ba29-83e2c5178caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:16.223117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247d89a-6faa-4233-ba29-83e2c5178caa None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:16.223117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247d89a-6faa-4233-ba29-83e2c5178caa None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:16.223117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247d89a-6faa-4233-ba29-83e2c5178caa None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:16.223605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247d89a-6faa-4233-ba29-83e2c5178caa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:16.223605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247d89a-6faa-4233-ba29-83e2c5178caa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:16.223605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247d89a-6faa-4233-ba29-83e2c5178caa None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:16.235261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 613/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:39:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:16.241236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:16.327892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:16.367503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c6f98-7e3a-4986-b770-2d6826ab5a76 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:16.404663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb5e09-b0b0-4ef4-a04f-e3fe76e1bb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:16.405874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5e09-b0b0-4ef4-a04f-e3fe76e1bb47 None admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:16.406196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5e09-b0b0-4ef4-a04f-e3fe76e1bb47 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:16.406522 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb5e09-b0b0-4ef4-a04f-e3fe76e1bb47 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/8957fc12ab74474f890b9188ba632fab/roles/af374fa10d7d4a7aa378572855842eb0` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:16.423579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb5e09-b0b0-4ef4-a04f-e3fe76e1bb47 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=8957fc12ab74474f890b9188ba632fab, role_id=af374fa10d7d4a7aa378572855842eb0)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:16.425237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb5e09-b0b0-4ef4-a04f-e3fe76e1bb47 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:16.441410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 619/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:39:16 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/8957fc12ab74474f890b9188ba632fab/roles/af374fa10d7d4a7aa378572855842eb0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:39:18.775286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8b512-1fe2-46df-b7fb-78f629790fcb None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:18.775862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8b512-1fe2-46df-b7fb-78f629790fcb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:18.775862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8b512-1fe2-46df-b7fb-78f629790fcb None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:18.777225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 614/1233] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:18.783363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4973-d65b-4e5e-8d24-d356478fb86d None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:18.784066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4973-d65b-4e5e-8d24-d356478fb86d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:18.784066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4973-d65b-4e5e-8d24-d356478fb86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:18.802388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bfda4973-d65b-4e5e-8d24-d356478fb86d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:18.803567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bfda4973-d65b-4e5e-8d24-d356478fb86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:18.873447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 620/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:19.142906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8b512-1fe2-46df-b7fb-78f629790fcb None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:19.169051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:19.264674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8b512-1fe2-46df-b7fb-78f629790fcb None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:19.302842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e86cf-feb8-4b1f-9bf6-a5678de8f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:19.303474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e86cf-feb8-4b1f-9bf6-a5678de8f4ba None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:19.303756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e86cf-feb8-4b1f-9bf6-a5678de8f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:19.304029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e86cf-feb8-4b1f-9bf6-a5678de8f4ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:19.316633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e86cf-feb8-4b1f-9bf6-a5678de8f4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-a70e86cf-feb8-4b1f-9bf6-a5678de8f4ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:19.322539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:19.323534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:19.324395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 615/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Sep 13 19:39:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:19.330294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda4973-d65b-4e5e-8d24-d356478fb86d None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:19.379464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:19.437482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda4973-d65b-4e5e-8d24-d356478fb86d None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:19.475201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:19.475811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:19.476081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:19.476345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:19.477010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] RBAC: Adding query filter params (name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:19.477301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:19.478721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:19.487671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 621/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Sep 13 19:39:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:19.495262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e86cf-feb8-4b1f-9bf6-a5678de8f4ba None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:19.589117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:19.607517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e86cf-feb8-4b1f-9bf6-a5678de8f4ba None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:19.648718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:19.649355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:19.650688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:19.650688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:19.661159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:19.662680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:19.678068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:19.679236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:19.680435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:19.680435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:19.680435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:19.680435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 616/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:39:19.685107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:19.794976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc50cc2-c7be-4d48-9489-53e64d0e7d46 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:19.805216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:19.841224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:19.842065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:19.842951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:19.843259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:19.844143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:19.844486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:19.846085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:19.861591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 622/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:19.867127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:19.988305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2d8cf-6a45-49b9-befd-62335f8d50f3 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:20.036428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:20.037073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:20.037354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:20.037691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] PATH_INFO: `/v3/users/placement` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:20.040743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:20.048105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:20.049982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:39:20.064118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:20.066263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:20.067864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 617/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:20.073093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:20.212696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbacef6-c7a5-4745-8743-664c56e733be None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:20.265408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:20.266392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:20.266902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:20.267285 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:20.268209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:20.268613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:20.270383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:20.298004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 623/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 13 19:39:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:20.304716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:20.418183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f79da3-8696-4ebc-88a1-8872072e8f6b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:20.458969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351e272-f265-4f48-9686-ae9ee4168e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:20.459592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351e272-f265-4f48-9686-ae9ee4168e1c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:20.459893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351e272-f265-4f48-9686-ae9ee4168e1c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:20.460149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351e272-f265-4f48-9686-ae9ee4168e1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:20.532756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:20.543648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351e272-f265-4f48-9686-ae9ee4168e1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:20.547036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351e272-f265-4f48-9686-ae9ee4168e1c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:20.666547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 618/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:20.673420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:20.708488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:20.798023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4a6dd-51c9-41ab-bdb4-49db8957978c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:20.835115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:20.835878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:20.836155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:20.836423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:20.848418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:20.850022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:20.867229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:20.868669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:20.869635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:20.869635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:20.869635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:20.869635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 624/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:20.874591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351e272-f265-4f48-9686-ae9ee4168e1c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:20.986019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:20.993859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351e272-f265-4f48-9686-ae9ee4168e1c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:21.033822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:21.034509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:21.034822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:21.035140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:21.035861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:21.036213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:21.037753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:21.051274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 619/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:21.057576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:21.161711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:21.186765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed635636-d75b-4979-8c46-cee668d585b7 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:21.224668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:21.225323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:21.225623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:21.225903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:21.233273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:21.234854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:39:21.247826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:21.249497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:21.250700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 625/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:39:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:21.255956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:21.370176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbd617e-eadb-48a7-a2aa-d2ec56da12c7 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:21.414350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654aff81-7c4c-478f-aafb-083949dd56b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:21.415129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aff81-7c4c-478f-aafb-083949dd56b4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:21.415534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aff81-7c4c-478f-aafb-083949dd56b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:21.415947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654aff81-7c4c-478f-aafb-083949dd56b4 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:21.416738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654aff81-7c4c-478f-aafb-083949dd56b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:21.417053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654aff81-7c4c-478f-aafb-083949dd56b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:21.418706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654aff81-7c4c-478f-aafb-083949dd56b4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:21.434074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 620/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:39:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:21.441777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:21.574078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76257276-8e01-4ea8-b604-6534701a1d40 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:21.621172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25cb01-1c35-47dc-b37a-86119d597679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:21.621871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cb01-1c35-47dc-b37a-86119d597679 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:21.622193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cb01-1c35-47dc-b37a-86119d597679 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:21.622408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cb01-1c35-47dc-b37a-86119d597679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:21.624247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25cb01-1c35-47dc-b37a-86119d597679 None admin] RBAC: Adding query filter params (role.id=af374fa10d7d4a7aa378572855842eb0, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=8957fc12ab74474f890b9188ba632fab) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:21.627090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25cb01-1c35-47dc-b37a-86119d597679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:21.628715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25cb01-1c35-47dc-b37a-86119d597679 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:21.636157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 626/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:21 2022] GET /identity/v3/role_assignments?user.id=8957fc12ab74474f890b9188ba632fab&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=af374fa10d7d4a7aa378572855842eb0 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:24.050032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6a29f-146e-4f2f-81de-29e0d7bfe8a3 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:24.050602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6a29f-146e-4f2f-81de-29e0d7bfe8a3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:24.050602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6a29f-146e-4f2f-81de-29e0d7bfe8a3 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:24.051972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 621/1247] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:24.061524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d12f-7db2-44df-aa61-d220258b7ceb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:24.061524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d12f-7db2-44df-aa61-d220258b7ceb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:24.061524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1d12f-7db2-44df-aa61-d220258b7ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:24.066101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:24.080791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-04e1d12f-7db2-44df-aa61-d220258b7ceb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:24.082284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-04e1d12f-7db2-44df-aa61-d220258b7ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:24.160508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 627/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:24.432500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6a29f-146e-4f2f-81de-29e0d7bfe8a3 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:24.455033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:24.563452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6a29f-146e-4f2f-81de-29e0d7bfe8a3 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:24.611598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d9d54-cea4-43d3-ab66-7bdee0fed880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:24.612254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d9d54-cea4-43d3-ab66-7bdee0fed880 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:24.612535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d9d54-cea4-43d3-ab66-7bdee0fed880 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:24.612805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d9d54-cea4-43d3-ab66-7bdee0fed880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:24.624338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d9d54-cea4-43d3-ab66-7bdee0fed880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-759d9d54-cea4-43d3-ab66-7bdee0fed880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:24.629554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:24.630505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:24.631369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 622/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:24.636826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1d12f-7db2-44df-aa61-d220258b7ceb None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:24.747939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1d12f-7db2-44df-aa61-d220258b7ceb None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:24.781354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:24.787320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:24.787947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:24.788238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:24.788510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:24.789325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:24.789651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:24.791281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:24.801937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 628/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:24.807617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d9d54-cea4-43d3-ab66-7bdee0fed880 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:24.905968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:24.936879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d9d54-cea4-43d3-ab66-7bdee0fed880 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:24.977796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:24.978470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:24.978755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:24.979022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:24.991295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:24.992804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:25.010238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:25.011317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:25.012220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:25.012220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:25.012220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:25.012220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 623/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:25.018163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:25.138027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c8791-e656-4ef6-90d6-441f29c7b722 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:25.173312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:25.174374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:25.174730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:25.175090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:25.175834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:25.176143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:25.177766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:25.194069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 629/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:25.199818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:25.317907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29def4-63d0-409b-9c03-db9225ac5ef8 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:25.348931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:25.359225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:25.359869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:25.360212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:25.360407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:25.368465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:25.369963 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:39:25.382401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:25.383664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:25.384649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 624/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:25.389742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:25.457934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:25.506904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc56ffd-2a76-4a2f-a495-5906d8bbed66 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:25.547012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:25.547648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:25.547927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:25.548238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:25.549190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:25.549523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:25.551030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:25.570019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 630/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 13 19:39:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:25.575767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:25.691718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdddc2-c210-4a7e-bb74-3edcca9a61c2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:25.727938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc3fef-8a8b-43c8-87e4-4c5be0105665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:25.728631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc3fef-8a8b-43c8-87e4-4c5be0105665 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:25.728914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc3fef-8a8b-43c8-87e4-4c5be0105665 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:25.729189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc3fef-8a8b-43c8-87e4-4c5be0105665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:25.788970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:25.803274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc3fef-8a8b-43c8-87e4-4c5be0105665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:25.804738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc3fef-8a8b-43c8-87e4-4c5be0105665 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:25.912744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 625/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:25.919949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:26.042444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b5b82-93d3-406d-916e-758298ff49f9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:26.055750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:26.082512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:26.084441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:26.084858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:26.085308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:26.098170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:26.099618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:26.115734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:26.116919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:26.117996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:26.117996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:26.117996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:26.117996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 631/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:39:26.123739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc3fef-8a8b-43c8-87e4-4c5be0105665 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:26.223328 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:26.237200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc3fef-8a8b-43c8-87e4-4c5be0105665 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:26.274808 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:26.275430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:26.275710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:26.275982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:26.276659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:26.276947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:26.278380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:26.291188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 626/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:26.297206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:26.401854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6c56-052c-4b55-8635-44cce4836b2c None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:26.437843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:26.438495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:26.438773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:26.439093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:26.446397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:26.447859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:39:26.460002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:26.461185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:26.462173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 632/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:39:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:26.467965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:26.587793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081af93-cb26-4247-9ef8-17fb55cb2c9c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:26.622468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc216b0-5c0b-45df-bcfa-e08e8b1f5d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:26.623201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc216b0-5c0b-45df-bcfa-e08e8b1f5d77 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:26.623483 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc216b0-5c0b-45df-bcfa-e08e8b1f5d77 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:26.623868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc216b0-5c0b-45df-bcfa-e08e8b1f5d77 None admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:26.624523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc216b0-5c0b-45df-bcfa-e08e8b1f5d77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:26.624829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc216b0-5c0b-45df-bcfa-e08e8b1f5d77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:26.626293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc216b0-5c0b-45df-bcfa-e08e8b1f5d77 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:26.639141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 627/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:39:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:26.645170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:26.671611 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:26.753739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad647384-3048-4edc-a446-95f33fa9de87 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:26.789213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afeca8-6f84-4701-82fb-d83b41234a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:26.789853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afeca8-6f84-4701-82fb-d83b41234a8d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:26.790135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afeca8-6f84-4701-82fb-d83b41234a8d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:26.790405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afeca8-6f84-4701-82fb-d83b41234a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:26.791129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afeca8-6f84-4701-82fb-d83b41234a8d None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=8957fc12ab74474f890b9188ba632fab) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:26.791434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afeca8-6f84-4701-82fb-d83b41234a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:26.792849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afeca8-6f84-4701-82fb-d83b41234a8d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:26.798498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 633/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:26 2022] GET /identity/v3/role_assignments?user.id=8957fc12ab74474f890b9188ba632fab&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=b698800ceec3460a8519ea627650da0f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:28.904421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac7f21-e905-4e80-9cc9-8188508142b9 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:28.904998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac7f21-e905-4e80-9cc9-8188508142b9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:28.904998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac7f21-e905-4e80-9cc9-8188508142b9 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:28.906421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 628/1261] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:28.911550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df633cf-5bce-4da5-84fa-9c5919d8fd5b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:28.911983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df633cf-5bce-4da5-84fa-9c5919d8fd5b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:28.912088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df633cf-5bce-4da5-84fa-9c5919d8fd5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:28.930370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5df633cf-5bce-4da5-84fa-9c5919d8fd5b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:28.931543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5df633cf-5bce-4da5-84fa-9c5919d8fd5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:28.994398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 634/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:29.258466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac7f21-e905-4e80-9cc9-8188508142b9 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:29.269559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:29.357798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac7f21-e905-4e80-9cc9-8188508142b9 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:29.390601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e0cea-a2d6-49f5-bfcf-79636502fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:29.391426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e0cea-a2d6-49f5-bfcf-79636502fa79 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:29.391909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e0cea-a2d6-49f5-bfcf-79636502fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:29.392372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e0cea-a2d6-49f5-bfcf-79636502fa79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:29.404004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e0cea-a2d6-49f5-bfcf-79636502fa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-ec4e0cea-a2d6-49f5-bfcf-79636502fa79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:29.408945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:29.410091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:29.411210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 629/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:29.416059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df633cf-5bce-4da5-84fa-9c5919d8fd5b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:29.433593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:29.519099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df633cf-5bce-4da5-84fa-9c5919d8fd5b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:29.555175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:29.555882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:29.556311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:29.556495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:29.557076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:29.557374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:29.558818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:29.568302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 635/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:29.574087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e0cea-a2d6-49f5-bfcf-79636502fa79 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:29.643862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:29.675890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e0cea-a2d6-49f5-bfcf-79636502fa79 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:29.709014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:29.709663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:29.709947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:29.710218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:29.719916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:29.721350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:29.735867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:29.736980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:29.737957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:29.737957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:29.737957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:29.737957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 630/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:29.742627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:29.821102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:29.839526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f853d9-dfd4-4077-b56a-b81c8b8be477 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:29.871386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:29.871972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:29.872245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:29.872513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:29.873710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:29.874009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:29.875353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:29.886662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 636/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:29.892297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:29.989577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b652c39-5969-46ef-bc11-d054c08194e2 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:30.009298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:30.026326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:30.026953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:30.027229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:30.027500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:30.035847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:30.037273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:39:30.048861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:30.050036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:30.051170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 631/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:30.055744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:30.152125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4f5b3-4a55-4d90-88dc-8efccd0f4b8d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:30.185006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:30.186284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:30.186559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:30.186830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:30.187545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:30.187848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:30.189247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:30.197853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:30.206316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 637/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 13 19:39:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:30.212032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:30.308836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ba586-7f39-4399-9aa2-0a4acd6ea184 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:30.341244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:30.342398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:30.342678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:30.342944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:30.353042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:30.354494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:30.368521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:30.369649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:30.370886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:30.370886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:30.370886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:30.370886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 632/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:30.375345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:30.472247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0c132-57ac-48ee-bb32-88390d121da9 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:30.513855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:30.514112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:30.514220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:30.514863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:30.515639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:30.515969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:30.517546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:30.531132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 638/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:30.536774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:30.574199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:30.641556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbe059-bcef-435c-b68c-322122726b6d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:30.674916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c300120-298f-484d-84e5-74268229143c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:30.675530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c300120-298f-484d-84e5-74268229143c None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:30.675812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c300120-298f-484d-84e5-74268229143c None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:30.676087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c300120-298f-484d-84e5-74268229143c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:30.682481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c300120-298f-484d-84e5-74268229143c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:30.683910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c300120-298f-484d-84e5-74268229143c None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-3c300120-298f-484d-84e5-74268229143c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:39:30.695135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:30.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:30.697022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 633/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:39:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:30.702254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:30.758903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:30.799629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49a56f8-f44c-45b3-a2ca-f643b805bc09 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:30.831947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8451b820-cb32-4f58-8626-11804b979f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:30.832543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451b820-cb32-4f58-8626-11804b979f37 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:30.832820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451b820-cb32-4f58-8626-11804b979f37 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:30.833093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451b820-cb32-4f58-8626-11804b979f37 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:30.833799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451b820-cb32-4f58-8626-11804b979f37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:30.834107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451b820-cb32-4f58-8626-11804b979f37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:30.835523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8451b820-cb32-4f58-8626-11804b979f37 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:30.847822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 639/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:39:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:30.855095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c300120-298f-484d-84e5-74268229143c None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:30.942856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:30.952403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c300120-298f-484d-84e5-74268229143c None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:30.984941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f587c62-2114-4223-bc3e-b156c4e70229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:30.985584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587c62-2114-4223-bc3e-b156c4e70229 None admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:30.985897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587c62-2114-4223-bc3e-b156c4e70229 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:30.986171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f587c62-2114-4223-bc3e-b156c4e70229 None admin] PATH_INFO: `/v3/projects/4c5e1753942b401696ad633361fa21c9/users/8957fc12ab74474f890b9188ba632fab/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:30.988019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f587c62-2114-4223-bc3e-b156c4e70229 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e1753942b401696ad633361fa21c9, user_id=8957fc12ab74474f890b9188ba632fab, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:30.989448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f587c62-2114-4223-bc3e-b156c4e70229 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:31.001488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 634/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Sep 13 19:39:30 2022] PUT /identity/v3/projects/4c5e1753942b401696ad633361fa21c9/users/8957fc12ab74474f890b9188ba632fab/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:39:33.109905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be710f-9185-46e8-80f6-c9b5e750087f None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:33.110433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be710f-9185-46e8-80f6-c9b5e750087f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:33.110433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be710f-9185-46e8-80f6-c9b5e750087f None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:33.111788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 640/1274] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:33.116803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21f58-ca09-484b-9bc5-f9e7f6b6fcbd None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:33.117209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21f58-ca09-484b-9bc5-f9e7f6b6fcbd None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:33.117347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21f58-ca09-484b-9bc5-f9e7f6b6fcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:33.135822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-21a21f58-ca09-484b-9bc5-f9e7f6b6fcbd None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:33.137005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-21a21f58-ca09-484b-9bc5-f9e7f6b6fcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:33.202471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 635/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:33.470984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be710f-9185-46e8-80f6-c9b5e750087f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:33.580743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be710f-9185-46e8-80f6-c9b5e750087f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:33.598041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:33.614073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de76bcf-9d2e-4ae0-a396-4e0f3d31d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:33.614687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de76bcf-9d2e-4ae0-a396-4e0f3d31d1dc None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:33.614959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de76bcf-9d2e-4ae0-a396-4e0f3d31d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:33.615225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de76bcf-9d2e-4ae0-a396-4e0f3d31d1dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:33.625305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de76bcf-9d2e-4ae0-a396-4e0f3d31d1dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-6de76bcf-9d2e-4ae0-a396-4e0f3d31d1dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:33.630086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:33.631215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:33.632424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 641/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:33.637117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a21f58-ca09-484b-9bc5-f9e7f6b6fcbd None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:33.716339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:33.735174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a21f58-ca09-484b-9bc5-f9e7f6b6fcbd None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:33.770493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:33.771093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:33.771372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:33.771640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:33.772295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:33.772597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:33.773978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:33.781791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 636/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:33.787584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de76bcf-9d2e-4ae0-a396-4e0f3d31d1dc None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:33.884368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de76bcf-9d2e-4ae0-a396-4e0f3d31d1dc None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:33.916095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:33.916702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:33.916978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:33.917248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:33.927835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:33.929269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:33.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:33.944864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:33.945896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:33.945896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:33.945896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:33.945896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 642/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:33.950417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:34.047025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494304-f9fe-4ba5-b991-b62d75cee293 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:34.079380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:34.079967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.080243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.080511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.081138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:34.081437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:34.082846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:34.094623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 637/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:34.100223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:34.130307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:34.196614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75264ac3-9a54-42ee-a016-af331e6f6370 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:34.228937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:34.229561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.229941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.230213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.238033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:34.239451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:39:34.250366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:34.251490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:34.252482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 643/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:34.257207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:34.260965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:34.354507 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf1d03-7380-40e0-b5a3-f34b7dff6a19 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:34.387297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:34.387888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.388215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.388485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.389197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:34.389500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:34.391014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:34.407858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 638/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Sep 13 19:39:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:34.413520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:34.507874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:34.517769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228af96-0773-47f6-b110-34dd238c40fa None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:34.550096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155f604-53fa-4fa2-9278-c783bfa9cb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:34.550693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f604-53fa-4fa2-9278-c783bfa9cb1d None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.550970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f604-53fa-4fa2-9278-c783bfa9cb1d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.551240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155f604-53fa-4fa2-9278-c783bfa9cb1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.615884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155f604-53fa-4fa2-9278-c783bfa9cb1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:34.617327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155f604-53fa-4fa2-9278-c783bfa9cb1d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:34.702030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:34.712207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 644/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:34.718116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:34.777962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:34.816028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6530d-115e-4fed-a86a-6cb02e50f28b None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:34.848827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:34.849441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.849762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.850036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.860212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:34.861680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:34.875824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:34.877053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:39:34.878362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:34.878362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:39:34.878362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:34.878362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 639/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:39:34.882546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155f604-53fa-4fa2-9278-c783bfa9cb1d None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:34.977560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155f604-53fa-4fa2-9278-c783bfa9cb1d None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:35.009948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:35.010549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:35.010828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:35.011100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:35.011736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:35.012036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:35.013433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:35.025287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 645/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Sep 13 19:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:35.031320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:35.128690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42d286e-d984-4bf6-8322-2d302313e75d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:35.141578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:35.161735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:35.162345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:35.162623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:35.162892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:35.169237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:35.171269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:39:35.182935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:35.183939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 19:39:35.184910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 640/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Sep 13 19:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:35.189977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:35.238865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:35.287588 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c202-9617-4f02-a7fc-c96589e0b9c5 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:35.319443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355580f-4cf6-40c0-8c00-523eff7546e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:35.320032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355580f-4cf6-40c0-8c00-523eff7546e7 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:35.320308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355580f-4cf6-40c0-8c00-523eff7546e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:35.320576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355580f-4cf6-40c0-8c00-523eff7546e7 None admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:35.321250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355580f-4cf6-40c0-8c00-523eff7546e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:35.321550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355580f-4cf6-40c0-8c00-523eff7546e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:35.322985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355580f-4cf6-40c0-8c00-523eff7546e7 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:35.335951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 646/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Sep 13 19:39:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:35.342099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:35.440170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a8bd1-3ae9-46a9-90c7-b6cb2cf8cc7d None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:35.475023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe94f440-b057-4241-9c9d-25b179b98d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:35.475615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94f440-b057-4241-9c9d-25b179b98d6b None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:35.475895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94f440-b057-4241-9c9d-25b179b98d6b None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:35.476168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe94f440-b057-4241-9c9d-25b179b98d6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:35.476879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94f440-b057-4241-9c9d-25b179b98d6b None admin] RBAC: Adding query filter params (role.id=b698800ceec3460a8519ea627650da0f, scope.project.id=4c5e1753942b401696ad633361fa21c9, user.id=8957fc12ab74474f890b9188ba632fab) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:35.477186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94f440-b057-4241-9c9d-25b179b98d6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:35.478644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe94f440-b057-4241-9c9d-25b179b98d6b None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:35.484338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 641/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:35 2022] GET /identity/v3/role_assignments?user.id=8957fc12ab74474f890b9188ba632fab&scope.project.id=4c5e1753942b401696ad633361fa21c9&role.id=b698800ceec3460a8519ea627650da0f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:37.632143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dd40-6ed5-4c56-ae0e-047d59c23657 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:37.632742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dd40-6ed5-4c56-ae0e-047d59c23657 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:37.632742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0dd40-6ed5-4c56-ae0e-047d59c23657 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:37.634054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 647/1288] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:37.639556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536dae4-ed98-460e-8d51-1f1e745b48cf None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:37.639994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536dae4-ed98-460e-8d51-1f1e745b48cf None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:37.640134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536dae4-ed98-460e-8d51-1f1e745b48cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:37.658590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-d536dae4-ed98-460e-8d51-1f1e745b48cf None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:37.659770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d536dae4-ed98-460e-8d51-1f1e745b48cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:37.691024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:37.730481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 642/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:37.996241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0dd40-6ed5-4c56-ae0e-047d59c23657 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:38.100204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0dd40-6ed5-4c56-ae0e-047d59c23657 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:38.111064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:38.133818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:38.134442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:38.134721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:38.134996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] PATH_INFO: `/v3/services/placement` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:38.135578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:38.136929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:38.145566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:38.146862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:38.147815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 648/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:39:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:38.152610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536dae4-ed98-460e-8d51-1f1e745b48cf None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:38.249224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536dae4-ed98-460e-8d51-1f1e745b48cf None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:38.282830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61e7b5-c266-4b25-aeb4-8c97c4fa40f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:38.283426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61e7b5-c266-4b25-aeb4-8c97c4fa40f4 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:38.283710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61e7b5-c266-4b25-aeb4-8c97c4fa40f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:38.283985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61e7b5-c266-4b25-aeb4-8c97c4fa40f4 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:38.284606 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61e7b5-c266-4b25-aeb4-8c97c4fa40f4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:38.284912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61e7b5-c266-4b25-aeb4-8c97c4fa40f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:38.286296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61e7b5-c266-4b25-aeb4-8c97c4fa40f4 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:38.290928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 643/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Sep 13 19:39:38 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:38.296524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:38.394290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c87734-2396-4888-84d7-ae4f11899d2a None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:38.426601 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a24b7-ea06-4c5b-9b3c-a5ee5e8db8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:38.427187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a24b7-ea06-4c5b-9b3c-a5ee5e8db8de None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:38.427466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a24b7-ea06-4c5b-9b3c-a5ee5e8db8de None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:38.427736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a24b7-ea06-4c5b-9b3c-a5ee5e8db8de None admin] PATH_INFO: `/v3/services` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:38.428355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a24b7-ea06-4c5b-9b3c-a5ee5e8db8de None admin] RBAC: Adding query filter params (type=placement) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:38.428657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a24b7-ea06-4c5b-9b3c-a5ee5e8db8de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:38.430035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a24b7-ea06-4c5b-9b3c-a5ee5e8db8de None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:38.434288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 649/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Sep 13 19:39:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:40.523904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae6bf6-ef1c-4165-a0a1-48acff100881 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:40.524471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae6bf6-ef1c-4165-a0a1-48acff100881 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:40.524471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae6bf6-ef1c-4165-a0a1-48acff100881 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:40.525819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 644/1293] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:40.532102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401525d-4055-4665-946d-c3c72342ed93 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:40.532579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401525d-4055-4665-946d-c3c72342ed93 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:40.532685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401525d-4055-4665-946d-c3c72342ed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:40.551012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e401525d-4055-4665-946d-c3c72342ed93 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:40.552205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e401525d-4055-4665-946d-c3c72342ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:40.581393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:40.617774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 650/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:40.879854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae6bf6-ef1c-4165-a0a1-48acff100881 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:40.910973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:40.980183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae6bf6-ef1c-4165-a0a1-48acff100881 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:41.014077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0ded9-2e80-46f5-b7f2-90ef742b31c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:41.014697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0ded9-2e80-46f5-b7f2-90ef742b31c3 None admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:41.015111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0ded9-2e80-46f5-b7f2-90ef742b31c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:41.015249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0ded9-2e80-46f5-b7f2-90ef742b31c3 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:41.015910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0ded9-2e80-46f5-b7f2-90ef742b31c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:41.017280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c0ded9-2e80-46f5-b7f2-90ef742b31c3 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:41.031389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 645/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Sep 13 19:39:40 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:43.138884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8723e3f-c0f3-432b-8429-865810b53dd7 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:43.139441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8723e3f-c0f3-432b-8429-865810b53dd7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:43.139441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8723e3f-c0f3-432b-8429-865810b53dd7 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:43.140787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 651/1296] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:43.146015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d00cde-ecc0-41be-b5a4-84094ed4869a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:43.146439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d00cde-ecc0-41be-b5a4-84094ed4869a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:43.146572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d00cde-ecc0-41be-b5a4-84094ed4869a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:43.164782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-66d00cde-ecc0-41be-b5a4-84094ed4869a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:43.165989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-66d00cde-ecc0-41be-b5a4-84094ed4869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:43.236690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 646/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:43.505255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8723e3f-c0f3-432b-8429-865810b53dd7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:43.603648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8723e3f-c0f3-432b-8429-865810b53dd7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:43.636275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:43.636895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:43.637174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:43.637447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:43.638564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:43.639937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:43.648603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:43.649810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:43.650753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 652/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:39:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:43.656154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d00cde-ecc0-41be-b5a4-84094ed4869a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:43.673985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:43.754398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d00cde-ecc0-41be-b5a4-84094ed4869a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:43.788686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:43.789233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:43.789513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:43.789818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:43.790441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:43.790744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:43.792107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:43.797184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 647/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Sep 13 19:39:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:43.803176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:43.834352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:43.901429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48d315-7157-4064-b313-d4d22cac2d65 None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:43.933031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95769f6c-72e8-428b-ba17-d962f721dc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:43.933648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769f6c-72e8-428b-ba17-d962f721dc62 None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:43.933934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769f6c-72e8-428b-ba17-d962f721dc62 None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:43.934201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95769f6c-72e8-428b-ba17-d962f721dc62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:43.935438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95769f6c-72e8-428b-ba17-d962f721dc62 None admin] RBAC: Adding query filter params (interface=public, service_id=aee80de22e004c69bfd72945f3343825, region_id=RegionOne) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:43.935744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95769f6c-72e8-428b-ba17-d962f721dc62 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:43.937097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95769f6c-72e8-428b-ba17-d962f721dc62 None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:43.942184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 653/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Sep 13 19:39:43 2022] GET /identity/v3/endpoints?service_id=aee80de22e004c69bfd72945f3343825&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:43.947779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:44.027420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:44.045591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced74d3-6ad1-4911-82ca-19486092c2a1 None admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:44.079224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07f0f0-c278-40c7-b3ab-eb5773f5e6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:44.079822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f0f0-c278-40c7-b3ab-eb5773f5e6f5 None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:44.080103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f0f0-c278-40c7-b3ab-eb5773f5e6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:44.080379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07f0f0-c278-40c7-b3ab-eb5773f5e6f5 None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:44.080969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07f0f0-c278-40c7-b3ab-eb5773f5e6f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:44.082357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07f0f0-c278-40c7-b3ab-eb5773f5e6f5 None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:44.088901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 648/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Sep 13 19:39:43 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:39:46.193218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a00922-5087-4d46-806c-bc9d589798a0 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:46.193786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a00922-5087-4d46-806c-bc9d589798a0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:46.193786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a00922-5087-4d46-806c-bc9d589798a0 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:46.195125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 654/1302] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:46.200168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740eb52-2dda-4055-a0e0-f95e7d4c5a32 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:46.200574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740eb52-2dda-4055-a0e0-f95e7d4c5a32 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:46.200721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740eb52-2dda-4055-a0e0-f95e7d4c5a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:46.218874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8740eb52-2dda-4055-a0e0-f95e7d4c5a32 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:46.220051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8740eb52-2dda-4055-a0e0-f95e7d4c5a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:46.284594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 649/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:46.549839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a00922-5087-4d46-806c-bc9d589798a0 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:46.640768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:46.648098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a00922-5087-4d46-806c-bc9d589798a0 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:46.680892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:46.681508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:46.681811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:46.682080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] PATH_INFO: `/v3/services/placement` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:46.682650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:46.683995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:46.693360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:46.694555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: ERROR keystone.server.flask.application Sep 13 19:39:46.695638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 655/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Sep 13 19:39:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:46.700586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740eb52-2dda-4055-a0e0-f95e7d4c5a32 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:46.776303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:46.799044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740eb52-2dda-4055-a0e0-f95e7d4c5a32 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:46.831713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a383f9a-7e08-48ed-b54a-5b658c0eccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:46.832248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a383f9a-7e08-48ed-b54a-5b658c0eccee None admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:46.832526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a383f9a-7e08-48ed-b54a-5b658c0eccee None admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:46.832798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a383f9a-7e08-48ed-b54a-5b658c0eccee None admin] PATH_INFO: `/v3/services` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:46.833416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a383f9a-7e08-48ed-b54a-5b658c0eccee None admin] RBAC: Adding query filter params (name=placement) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:46.833746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a383f9a-7e08-48ed-b54a-5b658c0eccee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:46.835108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a383f9a-7e08-48ed-b54a-5b658c0eccee None admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:46.840217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 650/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Sep 13 19:39:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:46.847034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:46.942205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7efa2-4068-47da-92db-733259737fee None admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:46.974031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67841a1-05ca-4503-acdc-d3fefab90f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:46.974614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67841a1-05ca-4503-acdc-d3fefab90f8d None admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:46.974890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67841a1-05ca-4503-acdc-d3fefab90f8d None admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:46.975158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67841a1-05ca-4503-acdc-d3fefab90f8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:46.975817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67841a1-05ca-4503-acdc-d3fefab90f8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:46.977157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67841a1-05ca-4503-acdc-d3fefab90f8d None admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:46.996114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 656/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:39:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:56.837241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=30 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:56.992479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:50.647332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0c1ee-5a17-4350-b5d7-56b50468da67 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:50.647801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0c1ee-5a17-4350-b5d7-56b50468da67 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:50.647801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0c1ee-5a17-4350-b5d7-56b50468da67 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:50.654124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 651/1307] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:40:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:50.655554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13873ff-258f-48b2-95c3-bffcb80179f6 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:50.656042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13873ff-258f-48b2-95c3-bffcb80179f6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:50.656266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13873ff-258f-48b2-95c3-bffcb80179f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:50.752862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d13873ff-258f-48b2-95c3-bffcb80179f6 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:50.754125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d13873ff-258f-48b2-95c3-bffcb80179f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:50.889217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 657/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:40:51.190689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81464e89-0804-42d5-a8c0-bcf95927c26b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:51.190689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81464e89-0804-42d5-a8c0-bcf95927c26b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:51.190689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81464e89-0804-42d5-a8c0-bcf95927c26b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:51.192641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 652/1309] 10.222.0.23 () {54 vars in 965 bytes} [Tue Sep 13 19:40:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:51.199797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb73a6b-907e-4e2d-9de2-61d7dfb3178a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:51.200106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb73a6b-907e-4e2d-9de2-61d7dfb3178a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:51.200363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb73a6b-907e-4e2d-9de2-61d7dfb3178a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:51.292549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fbb73a6b-907e-4e2d-9de2-61d7dfb3178a None None] MFA Rules not processed for user `9f77e57b2d96461c88e2fc33db22e22d`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:51.294163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fbb73a6b-907e-4e2d-9de2-61d7dfb3178a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:51.317952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:51.382763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 658/1310] 10.222.0.23 () {58 vars in 1062 bytes} [Tue Sep 13 19:40:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:40:51.391841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81464e89-0804-42d5-a8c0-bcf95927c26b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:51.393935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-81464e89-0804-42d5-a8c0-bcf95927c26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:51.508270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81464e89-0804-42d5-a8c0-bcf95927c26b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:51.588836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd9260-aa5c-416e-b794-7d8fd2179409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:51.589320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd9260-aa5c-416e-b794-7d8fd2179409 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:51.589599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd9260-aa5c-416e-b794-7d8fd2179409 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:51.589899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd9260-aa5c-416e-b794-7d8fd2179409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:51.591521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9cfd9260-aa5c-416e-b794-7d8fd2179409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:51.658545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:51.723927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd9260-aa5c-416e-b794-7d8fd2179409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:51.725415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd9260-aa5c-416e-b794-7d8fd2179409 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:51.747434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 653/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:40:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:40:54.189307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b34ef5-fc10-42dc-8a4c-abeaa8f1a4dc None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:54.189878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b34ef5-fc10-42dc-8a4c-abeaa8f1a4dc None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:54.189878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b34ef5-fc10-42dc-8a4c-abeaa8f1a4dc None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:54.191173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 659/1312] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:54.196822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246e002-fd1e-4c73-8c0b-d9ee76166dac None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:54.197341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246e002-fd1e-4c73-8c0b-d9ee76166dac None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:54.197341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246e002-fd1e-4c73-8c0b-d9ee76166dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:54.269986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7246e002-fd1e-4c73-8c0b-d9ee76166dac None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:54.271189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7246e002-fd1e-4c73-8c0b-d9ee76166dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:54.365753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 654/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:40:54.645879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b34ef5-fc10-42dc-8a4c-abeaa8f1a4dc None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:54.647429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d3b34ef5-fc10-42dc-8a4c-abeaa8f1a4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:54.752611 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b34ef5-fc10-42dc-8a4c-abeaa8f1a4dc None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:54.827689 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:54.830178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee3066-bb13-4fee-ad87-26316f7a6f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:54.830963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3066-bb13-4fee-ad87-26316f7a6f81 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:54.831279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3066-bb13-4fee-ad87-26316f7a6f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:54.831757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee3066-bb13-4fee-ad87-26316f7a6f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:54.833564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c8ee3066-bb13-4fee-ad87-26316f7a6f81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:54.942678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee3066-bb13-4fee-ad87-26316f7a6f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:54.944059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee3066-bb13-4fee-ad87-26316f7a6f81 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:54.961865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 660/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:40:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:40:57.423669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e31bf8-edc5-4a48-b7df-af97097a7a7b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:57.424216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e31bf8-edc5-4a48-b7df-af97097a7a7b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:57.424216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e31bf8-edc5-4a48-b7df-af97097a7a7b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:57.425542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 655/1315] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:57.431350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1580a-7879-43f6-a28f-27373eca7e90 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:57.431797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1580a-7879-43f6-a28f-27373eca7e90 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:57.431907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1580a-7879-43f6-a28f-27373eca7e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:57.495539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fab1580a-7879-43f6-a28f-27373eca7e90 None None] MFA Rules not processed for user `3678a202c2fa4669a16786db2586e75d`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:57.496725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fab1580a-7879-43f6-a28f-27373eca7e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:57.548085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:57.592325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 661/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:57 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:40:57.877575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e31bf8-edc5-4a48-b7df-af97097a7a7b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:57.879153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-71e31bf8-edc5-4a48-b7df-af97097a7a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:57.889410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:57.981989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e31bf8-edc5-4a48-b7df-af97097a7a7b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:58.054852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ca10a-4ed8-41db-b4fb-9e2299929b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:58.055807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ca10a-4ed8-41db-b4fb-9e2299929b76 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:58.056087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ca10a-4ed8-41db-b4fb-9e2299929b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:58.056358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ca10a-4ed8-41db-b4fb-9e2299929b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:58.057978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-372ca10a-4ed8-41db-b4fb-9e2299929b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:58.120618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:58.166401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ca10a-4ed8-41db-b4fb-9e2299929b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:58.167775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ca10a-4ed8-41db-b4fb-9e2299929b76 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:58.185928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 656/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:40:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:00.838976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd54018-a6c1-4ec7-915e-74de35997cd2 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:00.839528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd54018-a6c1-4ec7-915e-74de35997cd2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:00.839528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd54018-a6c1-4ec7-915e-74de35997cd2 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:00.841032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 662/1318] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:00.846606 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e8aac-9db1-4301-a8f3-b44905456398 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:00.847052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e8aac-9db1-4301-a8f3-b44905456398 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:00.847150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e8aac-9db1-4301-a8f3-b44905456398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:00.913049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f03e8aac-9db1-4301-a8f3-b44905456398 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:00.914326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f03e8aac-9db1-4301-a8f3-b44905456398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:01.010018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 657/1319] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:01.296714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd54018-a6c1-4ec7-915e-74de35997cd2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:01.298283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-efd54018-a6c1-4ec7-915e-74de35997cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:01.406632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd54018-a6c1-4ec7-915e-74de35997cd2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:01.484912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c06043-88c5-49be-96a0-8192c7379435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:01.485518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c06043-88c5-49be-96a0-8192c7379435 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:01.485827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c06043-88c5-49be-96a0-8192c7379435 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:01.486107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c06043-88c5-49be-96a0-8192c7379435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:01.487703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f0c06043-88c5-49be-96a0-8192c7379435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:01.495114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:01.599162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c06043-88c5-49be-96a0-8192c7379435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:01.600515 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c06043-88c5-49be-96a0-8192c7379435 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:01.621936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 663/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:04.409361 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b706e0c-27b1-41aa-9e56-c63f096ba6a0 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:04.409946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b706e0c-27b1-41aa-9e56-c63f096ba6a0 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:04.409946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b706e0c-27b1-41aa-9e56-c63f096ba6a0 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:04.411265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 658/1321] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:04.416630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8463431-ca08-4c6b-92e9-1ebbf1741ffb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:04.417202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8463431-ca08-4c6b-92e9-1ebbf1741ffb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:04.417202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8463431-ca08-4c6b-92e9-1ebbf1741ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:04.490463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-c8463431-ca08-4c6b-92e9-1ebbf1741ffb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:04.491613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c8463431-ca08-4c6b-92e9-1ebbf1741ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:04.554461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:04.595973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 664/1322] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:04.879033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b706e0c-27b1-41aa-9e56-c63f096ba6a0 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:04.880644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3b706e0c-27b1-41aa-9e56-c63f096ba6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:04.901057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:04.988788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b706e0c-27b1-41aa-9e56-c63f096ba6a0 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:05.068568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbeb272-ebe7-4be8-8cee-429119ac8f13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:05.069178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeb272-ebe7-4be8-8cee-429119ac8f13 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:05.069457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeb272-ebe7-4be8-8cee-429119ac8f13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:05.069763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeb272-ebe7-4be8-8cee-429119ac8f13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:05.071961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ecbeb272-ebe7-4be8-8cee-429119ac8f13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:05.143171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:05.180516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbeb272-ebe7-4be8-8cee-429119ac8f13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:05.181999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbeb272-ebe7-4be8-8cee-429119ac8f13 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:05.200764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 659/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:07.409931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6291388-00b8-4ce0-957b-d62187fd3cec None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:07.410509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6291388-00b8-4ce0-957b-d62187fd3cec None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:07.410509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6291388-00b8-4ce0-957b-d62187fd3cec None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:07.411848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 665/1324] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:07.417841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6febb-1986-4e92-aced-fce6e791ccfb None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:07.418262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6febb-1986-4e92-aced-fce6e791ccfb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:07.418518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6febb-1986-4e92-aced-fce6e791ccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:07.490521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-16a6febb-1986-4e92-aced-fce6e791ccfb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:07.491732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-16a6febb-1986-4e92-aced-fce6e791ccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:07.590082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 660/1325] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:07.887497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d1627-3de5-45b5-bcd3-ecdc9dcb1d53 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:07.887852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d1627-3de5-45b5-bcd3-ecdc9dcb1d53 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:07.888198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d1627-3de5-45b5-bcd3-ecdc9dcb1d53 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:07.889693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 666/1326] 10.222.0.23 () {54 vars in 965 bytes} [Tue Sep 13 19:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:07.895697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b072ab0-6691-4a67-a61f-9ebd6f681983 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:07.896083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b072ab0-6691-4a67-a61f-9ebd6f681983 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:07.896424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b072ab0-6691-4a67-a61f-9ebd6f681983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:07.927446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:07.973994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-4b072ab0-6691-4a67-a61f-9ebd6f681983 None None] MFA Rules not processed for user `9f77e57b2d96461c88e2fc33db22e22d`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:07.975220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4b072ab0-6691-4a67-a61f-9ebd6f681983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:08.055896 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 661/1327] 10.222.0.23 () {58 vars in 1062 bytes} [Tue Sep 13 19:41:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:41:08.065189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d1627-3de5-45b5-bcd3-ecdc9dcb1d53 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:08.066791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a26d1627-3de5-45b5-bcd3-ecdc9dcb1d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:08.170146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d1627-3de5-45b5-bcd3-ecdc9dcb1d53 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:08.243314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f2cae-6549-4f5c-8764-0ee056d92694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:08.243917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f2cae-6549-4f5c-8764-0ee056d92694 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:08.244191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f2cae-6549-4f5c-8764-0ee056d92694 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:08.244457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f2cae-6549-4f5c-8764-0ee056d92694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:08.246070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ff3f2cae-6549-4f5c-8764-0ee056d92694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:08.262497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:08.353719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f2cae-6549-4f5c-8764-0ee056d92694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:08.355153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f2cae-6549-4f5c-8764-0ee056d92694 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:08.373650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 667/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:10.793076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc2fc3-8f71-4430-9a45-72e157ef1373 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:10.793705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc2fc3-8f71-4430-9a45-72e157ef1373 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:10.793705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc2fc3-8f71-4430-9a45-72e157ef1373 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:10.794991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 662/1329] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:10.800479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ac41-edc3-47cc-8b11-6c692c491a8d None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:10.800995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ac41-edc3-47cc-8b11-6c692c491a8d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:10.800995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4ac41-edc3-47cc-8b11-6c692c491a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:10.869054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-aed4ac41-edc3-47cc-8b11-6c692c491a8d None None] MFA Rules not processed for user `3678a202c2fa4669a16786db2586e75d`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:10.870300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-aed4ac41-edc3-47cc-8b11-6c692c491a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:10.934423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:10.966339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 668/1330] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:10 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:11.248233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc2fc3-8f71-4430-9a45-72e157ef1373 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:11.249866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-23dc2fc3-8f71-4430-9a45-72e157ef1373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:11.356048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc2fc3-8f71-4430-9a45-72e157ef1373 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:11.377641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:11.432709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a62f60-de40-4a06-9930-7b1cbf75e7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:11.433324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a62f60-de40-4a06-9930-7b1cbf75e7df service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:11.433627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a62f60-de40-4a06-9930-7b1cbf75e7df service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:11.433903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a62f60-de40-4a06-9930-7b1cbf75e7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:11.435513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d6a62f60-de40-4a06-9930-7b1cbf75e7df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:11.557165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a62f60-de40-4a06-9930-7b1cbf75e7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:11.559099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a62f60-de40-4a06-9930-7b1cbf75e7df service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:11.580148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 663/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:11 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:14.110967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeac69-ce84-4c04-a261-f102eafa1e86 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:14.111564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeac69-ce84-4c04-a261-f102eafa1e86 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:14.111564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeac69-ce84-4c04-a261-f102eafa1e86 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:14.112934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 669/1332] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:14.118871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baedd7-e820-4b10-b001-97762665c639 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:14.119304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baedd7-e820-4b10-b001-97762665c639 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:14.119409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baedd7-e820-4b10-b001-97762665c639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:14.184870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:14.187435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-58baedd7-e820-4b10-b001-97762665c639 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:14.189429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-58baedd7-e820-4b10-b001-97762665c639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:14.283351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 664/1333] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:41:14.566782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeac69-ce84-4c04-a261-f102eafa1e86 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:14.568343 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3bfeac69-ce84-4c04-a261-f102eafa1e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:14.669256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeac69-ce84-4c04-a261-f102eafa1e86 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:14.741422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d87db-7970-4f67-b944-2fe630350885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:14.742637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d87db-7970-4f67-b944-2fe630350885 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:14.742913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d87db-7970-4f67-b944-2fe630350885 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:14.743180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d87db-7970-4f67-b944-2fe630350885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:14.744766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7b4d87db-7970-4f67-b944-2fe630350885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:14.771173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:14.852103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d87db-7970-4f67-b944-2fe630350885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:14.853447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d87db-7970-4f67-b944-2fe630350885 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:14.871800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 670/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:17.077162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9fc0e-91e9-445b-a505-a11cc87f033d None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:17.077687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9fc0e-91e9-445b-a505-a11cc87f033d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:17.077811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9fc0e-91e9-445b-a505-a11cc87f033d None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:17.079103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 665/1335] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:17.084311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c91702-9cc8-4c13-83d6-97a871eef324 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:17.084730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c91702-9cc8-4c13-83d6-97a871eef324 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:17.084869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c91702-9cc8-4c13-83d6-97a871eef324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:17.150693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a2c91702-9cc8-4c13-83d6-97a871eef324 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:17.151903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a2c91702-9cc8-4c13-83d6-97a871eef324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:17.225358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:17.248069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 671/1336] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:17.539059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9fc0e-91e9-445b-a505-a11cc87f033d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:17.540589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3ba9fc0e-91e9-445b-a505-a11cc87f033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:17.642914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9fc0e-91e9-445b-a505-a11cc87f033d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:17.672533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:17.716783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02939c66-9571-415a-b046-0467618d69b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:17.717390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939c66-9571-415a-b046-0467618d69b2 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:17.717697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939c66-9571-415a-b046-0467618d69b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:17.717967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939c66-9571-415a-b046-0467618d69b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:17.719551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-02939c66-9571-415a-b046-0467618d69b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:17.824514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02939c66-9571-415a-b046-0467618d69b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:17.825974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02939c66-9571-415a-b046-0467618d69b2 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:17.844170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 666/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:20.466778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11277aa-5573-41c6-a04f-4b639665cf63 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:20.467514 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11277aa-5573-41c6-a04f-4b639665cf63 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:20.467709 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11277aa-5573-41c6-a04f-4b639665cf63 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:20.469368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 672/1338] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:20.475372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c41f9-6f4e-412a-b47b-05dd5d8757a4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:20.475792 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c41f9-6f4e-412a-b47b-05dd5d8757a4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:20.475928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c41f9-6f4e-412a-b47b-05dd5d8757a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:20.544479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-bd5c41f9-6f4e-412a-b47b-05dd5d8757a4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:20.545710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bd5c41f9-6f4e-412a-b47b-05dd5d8757a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:20.553862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:20.640998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 667/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:20.919172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11277aa-5573-41c6-a04f-4b639665cf63 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:20.920702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b11277aa-5573-41c6-a04f-4b639665cf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:21.022470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11277aa-5573-41c6-a04f-4b639665cf63 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:21.094405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c714f-4f79-4f8e-ab12-b674c3c8ffb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:21.095003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c714f-4f79-4f8e-ab12-b674c3c8ffb3 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:21.095277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c714f-4f79-4f8e-ab12-b674c3c8ffb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:21.095569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c714f-4f79-4f8e-ab12-b674c3c8ffb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:21.097137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1e0c714f-4f79-4f8e-ab12-b674c3c8ffb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:21.131596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:21.204500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c714f-4f79-4f8e-ab12-b674c3c8ffb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:21.205948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c714f-4f79-4f8e-ab12-b674c3c8ffb3 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:21.224751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 673/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:25.334446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9f8b-b661-48a0-a074-c82c84e18227 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:25.334977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9f8b-b661-48a0-a074-c82c84e18227 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:25.334977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d9f8b-b661-48a0-a074-c82c84e18227 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:25.336398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 668/1341] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:25.341697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6dcff-3190-441f-9c26-098515b5f14b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:25.342119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6dcff-3190-441f-9c26-098515b5f14b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:25.342259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6dcff-3190-441f-9c26-098515b5f14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:25.407439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e3d6dcff-3190-441f-9c26-098515b5f14b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:25.408579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e3d6dcff-3190-441f-9c26-098515b5f14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:25.494789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:25.506999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 674/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:25.791981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9f8b-b661-48a0-a074-c82c84e18227 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:25.793500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-132d9f8b-b661-48a0-a074-c82c84e18227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:25.899870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d9f8b-b661-48a0-a074-c82c84e18227 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:25.939494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:25.978349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88b85a-b388-43ed-bdf1-92306dacc671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:25.979245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88b85a-b388-43ed-bdf1-92306dacc671 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:25.979526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88b85a-b388-43ed-bdf1-92306dacc671 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:25.979799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88b85a-b388-43ed-bdf1-92306dacc671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:25.981385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3b88b85a-b388-43ed-bdf1-92306dacc671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:26.096836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88b85a-b388-43ed-bdf1-92306dacc671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:26.098249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88b85a-b388-43ed-bdf1-92306dacc671 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:26.117432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 669/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:41:28.895784 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02954d-4b1b-4951-b16e-ac88a288d28c None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:28.896355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02954d-4b1b-4951-b16e-ac88a288d28c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:28.896355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02954d-4b1b-4951-b16e-ac88a288d28c None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:28.897780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 675/1344] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:28.903002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6481-8ee1-4352-b7c2-e5be563b884a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:28.903466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6481-8ee1-4352-b7c2-e5be563b884a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:28.903571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd6481-8ee1-4352-b7c2-e5be563b884a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:28.969096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3bdd6481-8ee1-4352-b7c2-e5be563b884a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:28.970332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3bdd6481-8ee1-4352-b7c2-e5be563b884a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:28.987993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:29.068655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 670/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:29.351871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02954d-4b1b-4951-b16e-ac88a288d28c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:29.353526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4d02954d-4b1b-4951-b16e-ac88a288d28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:29.473457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02954d-4b1b-4951-b16e-ac88a288d28c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:29.558033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc691a-5c91-4832-ac1f-af237a7a0b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:29.558643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc691a-5c91-4832-ac1f-af237a7a0b18 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:29.558924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc691a-5c91-4832-ac1f-af237a7a0b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:29.559197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc691a-5c91-4832-ac1f-af237a7a0b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:29.560784 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a3bc691a-5c91-4832-ac1f-af237a7a0b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:29.604467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:29.672830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc691a-5c91-4832-ac1f-af237a7a0b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:29.674300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc691a-5c91-4832-ac1f-af237a7a0b18 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:29.692986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 676/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:33.805898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d177b-757d-4a19-bac2-f2af583b9e7e None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:33.806428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d177b-757d-4a19-bac2-f2af583b9e7e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:33.806428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d177b-757d-4a19-bac2-f2af583b9e7e None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:33.807899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 671/1347] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:33.813753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dab0ef-c25b-4a3e-8e1e-d4fb5e1f9bbb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:33.814204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dab0ef-c25b-4a3e-8e1e-d4fb5e1f9bbb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:33.814308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dab0ef-c25b-4a3e-8e1e-d4fb5e1f9bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:33.881850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e0dab0ef-c25b-4a3e-8e1e-d4fb5e1f9bbb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:33.883107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e0dab0ef-c25b-4a3e-8e1e-d4fb5e1f9bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:33.990844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:33.993239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 677/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:34.278713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d177b-757d-4a19-bac2-f2af583b9e7e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:34.280310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a43d177b-757d-4a19-bac2-f2af583b9e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:34.384658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d177b-757d-4a19-bac2-f2af583b9e7e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:34.432232 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:34.458773 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5700220-d20f-485b-90d9-5c6e8ae8ff14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:34.459378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5700220-d20f-485b-90d9-5c6e8ae8ff14 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:34.459651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5700220-d20f-485b-90d9-5c6e8ae8ff14 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:34.459926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5700220-d20f-485b-90d9-5c6e8ae8ff14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:34.461558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f5700220-d20f-485b-90d9-5c6e8ae8ff14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:34.570649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5700220-d20f-485b-90d9-5c6e8ae8ff14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:34.572423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5700220-d20f-485b-90d9-5c6e8ae8ff14 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:34.591594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 672/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:39.565058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b2b46-86a3-477e-a972-3c2442ebd4bb None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:39.565927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b2b46-86a3-477e-a972-3c2442ebd4bb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:39.566447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b2b46-86a3-477e-a972-3c2442ebd4bb None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:39.568332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 678/1350] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:39.574958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbca56a-66d5-468c-b3d8-6f31738f92ee None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:39.575396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbca56a-66d5-468c-b3d8-6f31738f92ee None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:39.575498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbca56a-66d5-468c-b3d8-6f31738f92ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:39.673494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8cbca56a-66d5-468c-b3d8-6f31738f92ee None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:39.673494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8cbca56a-66d5-468c-b3d8-6f31738f92ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:39.695345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:39.847582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 673/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:41:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:44.077206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39095139-1433-4c1d-8874-e54b6cf873e6 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:44.080027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39095139-1433-4c1d-8874-e54b6cf873e6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:44.080027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39095139-1433-4c1d-8874-e54b6cf873e6 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:44.080027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 679/1352] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:41:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:44.086425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9d03-b671-49ed-8541-c262d72d728b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:44.086904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9d03-b671-49ed-8541-c262d72d728b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:44.087259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e9d03-b671-49ed-8541-c262d72d728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:44.165720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8d7e9d03-b671-49ed-8541-c262d72d728b None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:44.166890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8d7e9d03-b671-49ed-8541-c262d72d728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:44.243195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:44.288096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 674/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:41:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:45.583440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f546a-9490-46cd-b659-d169947a28ec None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:45.584024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f546a-9490-46cd-b659-d169947a28ec None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:45.584024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f546a-9490-46cd-b659-d169947a28ec None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:45.585639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 680/1354] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:45.590632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d31a8-ebd3-402c-a38e-a2d54b758a9a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:45.591059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d31a8-ebd3-402c-a38e-a2d54b758a9a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:45.591411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d31a8-ebd3-402c-a38e-a2d54b758a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:45.667809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3f1d31a8-ebd3-402c-a38e-a2d54b758a9a None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:45.668985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3f1d31a8-ebd3-402c-a38e-a2d54b758a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:45.773039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 675/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:41:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:49.320365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bf098-402d-4550-8960-6a0d200811d7 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:49.320884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bf098-402d-4550-8960-6a0d200811d7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:49.320884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bf098-402d-4550-8960-6a0d200811d7 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:49.322181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 681/1356] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:49.328078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191b47-f5f1-4fe9-984b-2c59ea34c6f4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:49.328516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191b47-f5f1-4fe9-984b-2c59ea34c6f4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:49.328810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191b47-f5f1-4fe9-984b-2c59ea34c6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:49.369849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:49.405543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-28191b47-f5f1-4fe9-984b-2c59ea34c6f4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:49.406966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-28191b47-f5f1-4fe9-984b-2c59ea34c6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:49.509411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 676/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:49.818966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0c08-b713-4118-9eb8-4fe40634a31f None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:49.819467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0c08-b713-4118-9eb8-4fe40634a31f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:49.819897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115f0c08-b713-4118-9eb8-4fe40634a31f None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:49.821757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 682/1358] 10.222.0.23 () {54 vars in 962 bytes} [Tue Sep 13 19:41:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:49.829730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30024d38-c5f3-477b-a62f-a611bd8e70c6 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:49.830848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30024d38-c5f3-477b-a62f-a611bd8e70c6 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:49.831221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30024d38-c5f3-477b-a62f-a611bd8e70c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:49.926026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-30024d38-c5f3-477b-a62f-a611bd8e70c6 None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:49.927787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-30024d38-c5f3-477b-a62f-a611bd8e70c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:49.979833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:50.069935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f0c08-b713-4118-9eb8-4fe40634a31f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:50.071442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-115f0c08-b713-4118-9eb8-4fe40634a31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:50.073777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 677/1359] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Sep 13 19:41:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:50.228827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115f0c08-b713-4118-9eb8-4fe40634a31f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:50.305581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:50.347914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530e491-6eee-4f2c-82b8-6cb542ef6ab3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:50.348553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530e491-6eee-4f2c-82b8-6cb542ef6ab3 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:50.348804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530e491-6eee-4f2c-82b8-6cb542ef6ab3 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:50.349574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530e491-6eee-4f2c-82b8-6cb542ef6ab3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:50.351337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7530e491-6eee-4f2c-82b8-6cb542ef6ab3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:50.452600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530e491-6eee-4f2c-82b8-6cb542ef6ab3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:50.454236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7530e491-6eee-4f2c-82b8-6cb542ef6ab3 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:50.473241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 683/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:52.859412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed103c-6d42-4c68-af29-965d16ef8d31 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:52.860161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed103c-6d42-4c68-af29-965d16ef8d31 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:52.860161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed103c-6d42-4c68-af29-965d16ef8d31 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:52.861435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 678/1361] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:52.868037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6b2-bcdb-499e-8c50-e1b331300af5 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:52.868037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6b2-bcdb-499e-8c50-e1b331300af5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:52.868508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d6b2-bcdb-499e-8c50-e1b331300af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:52.941653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bda5d6b2-bcdb-499e-8c50-e1b331300af5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:52.943122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bda5d6b2-bcdb-499e-8c50-e1b331300af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:52.982692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:53.050183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 684/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:41:53.343788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed103c-6d42-4c68-af29-965d16ef8d31 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:53.345289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-61ed103c-6d42-4c68-af29-965d16ef8d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:53.525839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed103c-6d42-4c68-af29-965d16ef8d31 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:53.537572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:53.673840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2799da-3a69-43c7-b5a0-db9b65d4bf9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:53.674455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2799da-3a69-43c7-b5a0-db9b65d4bf9f service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:53.674734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2799da-3a69-43c7-b5a0-db9b65d4bf9f service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:53.675006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2799da-3a69-43c7-b5a0-db9b65d4bf9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:53.676592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3d2799da-3a69-43c7-b5a0-db9b65d4bf9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:53.772217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2799da-3a69-43c7-b5a0-db9b65d4bf9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:53.773797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2799da-3a69-43c7-b5a0-db9b65d4bf9f service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:53.792667 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 679/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:56.305255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374a0c-7e9e-4d0a-bda0-fcf2b1b9c0e5 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:56.305842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374a0c-7e9e-4d0a-bda0-fcf2b1b9c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:56.305842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c374a0c-7e9e-4d0a-bda0-fcf2b1b9c0e5 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:56.307102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 685/1364] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:56.312814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5a0be-7dc4-4b4a-b044-24edf7c86a64 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:56.313223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5a0be-7dc4-4b4a-b044-24edf7c86a64 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:56.313376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef5a0be-7dc4-4b4a-b044-24edf7c86a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:56.329831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:56.385116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2ef5a0be-7dc4-4b4a-b044-24edf7c86a64 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:56.386367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2ef5a0be-7dc4-4b4a-b044-24edf7c86a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:56.488211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 680/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:56.784110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7647ec-4997-480f-aa23-e15af437dc17 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:56.784422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7647ec-4997-480f-aa23-e15af437dc17 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:56.784702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7647ec-4997-480f-aa23-e15af437dc17 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:56.786185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 686/1366] 10.222.0.23 () {54 vars in 962 bytes} [Tue Sep 13 19:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:56.792165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17031c93-4978-47a4-96c9-d588f7e922ee None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:56.792458 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17031c93-4978-47a4-96c9-d588f7e922ee None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:56.792739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17031c93-4978-47a4-96c9-d588f7e922ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:56.873223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-17031c93-4978-47a4-96c9-d588f7e922ee None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:56.874809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-17031c93-4978-47a4-96c9-d588f7e922ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:56.912228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:56.998990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 681/1367] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Sep 13 19:41:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:57.007550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7647ec-4997-480f-aa23-e15af437dc17 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:57.009248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2b7647ec-4997-480f-aa23-e15af437dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:57.162859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7647ec-4997-480f-aa23-e15af437dc17 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:57.186250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:57.289699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cd74b-e4f2-41d8-96cc-3a6f90e8db83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:57.290312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cd74b-e4f2-41d8-96cc-3a6f90e8db83 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:57.290588 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cd74b-e4f2-41d8-96cc-3a6f90e8db83 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:57.290853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cd74b-e4f2-41d8-96cc-3a6f90e8db83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:57.292532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-561cd74b-e4f2-41d8-96cc-3a6f90e8db83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:57.391423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cd74b-e4f2-41d8-96cc-3a6f90e8db83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:57.393078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cd74b-e4f2-41d8-96cc-3a6f90e8db83 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:57.415799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 687/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:57.525348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392848e-9076-448c-a7ac-95bd864b7566 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:57.525348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392848e-9076-448c-a7ac-95bd864b7566 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:57.525348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392848e-9076-448c-a7ac-95bd864b7566 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:57.525348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 682/1369] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:57.526210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83ea99-359c-4846-8c17-43118e78e095 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:57.526506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83ea99-359c-4846-8c17-43118e78e095 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:57.526783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83ea99-359c-4846-8c17-43118e78e095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:57.565219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:57.611469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ef83ea99-359c-4846-8c17-43118e78e095 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:57.613101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ef83ea99-359c-4846-8c17-43118e78e095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:57.733973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 688/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:58.032078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664895a1-0109-49cd-a6f1-7315b15e8673 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:58.032665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664895a1-0109-49cd-a6f1-7315b15e8673 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:58.032947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664895a1-0109-49cd-a6f1-7315b15e8673 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:58.034493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 683/1371] 10.222.0.23 () {54 vars in 964 bytes} [Tue Sep 13 19:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:58.040210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24daccc-b03b-456b-ad5c-f5cda6a5433a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:58.040501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24daccc-b03b-456b-ad5c-f5cda6a5433a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:58.040778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24daccc-b03b-456b-ad5c-f5cda6a5433a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:58.119783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a24daccc-b03b-456b-ad5c-f5cda6a5433a None None] MFA Rules not processed for user `284740de651e4065845e945556a05ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:58.120999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a24daccc-b03b-456b-ad5c-f5cda6a5433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:58.172220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:58.208652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 689/1372] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Sep 13 19:41:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:58.215662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664895a1-0109-49cd-a6f1-7315b15e8673 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:58.217220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-664895a1-0109-49cd-a6f1-7315b15e8673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:58.335241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664895a1-0109-49cd-a6f1-7315b15e8673 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:58.383237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:58.431241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af8302-5ce9-4553-be47-7bc00a40a6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:58.431926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af8302-5ce9-4553-be47-7bc00a40a6ec service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:58.432267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af8302-5ce9-4553-be47-7bc00a40a6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:58.432605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af8302-5ce9-4553-be47-7bc00a40a6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:58.434291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f8af8302-5ce9-4553-be47-7bc00a40a6ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:58.564833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af8302-5ce9-4553-be47-7bc00a40a6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:58.566332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af8302-5ce9-4553-be47-7bc00a40a6ec service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:58.586083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 684/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:59.684470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b4a6-2ee3-467f-bf3f-8e982946ae12 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:59.685018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b4a6-2ee3-467f-bf3f-8e982946ae12 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:59.685018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b4a6-2ee3-467f-bf3f-8e982946ae12 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:59.686411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 690/1374] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:59.691935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c825ed-b79d-40f4-99ed-0ac5d2cec7c7 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:59.692371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c825ed-b79d-40f4-99ed-0ac5d2cec7c7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:59.692535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c825ed-b79d-40f4-99ed-0ac5d2cec7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:59.759303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e8c825ed-b79d-40f4-99ed-0ac5d2cec7c7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:59.760394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e8c825ed-b79d-40f4-99ed-0ac5d2cec7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:59.781272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:59.859800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 685/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:42:00.147898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b4a6-2ee3-467f-bf3f-8e982946ae12 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:00.149536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ddd6b4a6-2ee3-467f-bf3f-8e982946ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:00.309097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b4a6-2ee3-467f-bf3f-8e982946ae12 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:00.364407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:00.430321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978e30a-24ce-4726-a3b5-ef5e49b1622b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:00.430935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978e30a-24ce-4726-a3b5-ef5e49b1622b service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:00.431221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978e30a-24ce-4726-a3b5-ef5e49b1622b service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:00.431494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978e30a-24ce-4726-a3b5-ef5e49b1622b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:00.433110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c978e30a-24ce-4726-a3b5-ef5e49b1622b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:00.527131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978e30a-24ce-4726-a3b5-ef5e49b1622b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:00.528559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c978e30a-24ce-4726-a3b5-ef5e49b1622b service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:00.547537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 691/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:02.791505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a0445-e55d-48e7-9c68-2c7aba3e3716 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:02.792107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a0445-e55d-48e7-9c68-2c7aba3e3716 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:02.792365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96a0445-e55d-48e7-9c68-2c7aba3e3716 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:02.793902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 686/1377] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:42:02.799046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d857-c27d-41d1-8266-8b1e0e328c06 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:02.799465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d857-c27d-41d1-8266-8b1e0e328c06 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:02.799635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96d857-c27d-41d1-8266-8b1e0e328c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:02.872666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5d96d857-c27d-41d1-8266-8b1e0e328c06 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:02.873955 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5d96d857-c27d-41d1-8266-8b1e0e328c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:02.882505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:02.983538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 692/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:03.274470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a0445-e55d-48e7-9c68-2c7aba3e3716 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:03.275992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e96a0445-e55d-48e7-9c68-2c7aba3e3716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:03.429174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96a0445-e55d-48e7-9c68-2c7aba3e3716 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:03.433199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:03.548145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4f6c7-59c4-4892-8bd0-6cd6e1adf065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:03.548651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4f6c7-59c4-4892-8bd0-6cd6e1adf065 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:03.548934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4f6c7-59c4-4892-8bd0-6cd6e1adf065 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:03.549213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4f6c7-59c4-4892-8bd0-6cd6e1adf065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:03.550811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bcd4f6c7-59c4-4892-8bd0-6cd6e1adf065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:03.645518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4f6c7-59c4-4892-8bd0-6cd6e1adf065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:03.647163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4f6c7-59c4-4892-8bd0-6cd6e1adf065 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:03.665567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 687/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:05.871149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11201404-e5b5-4ced-8cac-178c1b91665b None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:05.871702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11201404-e5b5-4ced-8cac-178c1b91665b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:05.871702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11201404-e5b5-4ced-8cac-178c1b91665b None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:05.873202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 693/1380] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:05.878241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37100-6294-4022-88a4-543109219505 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:05.878666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37100-6294-4022-88a4-543109219505 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:05.878798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa37100-6294-4022-88a4-543109219505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:05.890904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:05.950219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-cfa37100-6294-4022-88a4-543109219505 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:05.951413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cfa37100-6294-4022-88a4-543109219505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:06.073820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 688/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:06.365565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11201404-e5b5-4ced-8cac-178c1b91665b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:06.367106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-11201404-e5b5-4ced-8cac-178c1b91665b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:06.519696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:06.532137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11201404-e5b5-4ced-8cac-178c1b91665b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:06.650100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205189c-32ed-4563-a3eb-5c5694c24b7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:06.650708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205189c-32ed-4563-a3eb-5c5694c24b7e service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:06.650982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205189c-32ed-4563-a3eb-5c5694c24b7e service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:06.651250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205189c-32ed-4563-a3eb-5c5694c24b7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:06.652896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d205189c-32ed-4563-a3eb-5c5694c24b7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:06.746996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205189c-32ed-4563-a3eb-5c5694c24b7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:06.748666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205189c-32ed-4563-a3eb-5c5694c24b7e service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:06.759707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:06.769429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 694/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:09.515171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452068-e533-4b6c-b37b-db87c13ad7c2 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:09.515688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452068-e533-4b6c-b37b-db87c13ad7c2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:09.515688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452068-e533-4b6c-b37b-db87c13ad7c2 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:09.516981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 689/1383] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:09.523074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7833eb-d050-4439-9c38-ba0f6b5d8bec None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:09.523597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7833eb-d050-4439-9c38-ba0f6b5d8bec None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:09.523704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7833eb-d050-4439-9c38-ba0f6b5d8bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:09.600745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ee7833eb-d050-4439-9c38-ba0f6b5d8bec None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:09.602077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ee7833eb-d050-4439-9c38-ba0f6b5d8bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:09.700567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 695/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:09.997275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b452068-e533-4b6c-b37b-db87c13ad7c2 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:09.999295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9b452068-e533-4b6c-b37b-db87c13ad7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:10.105339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:10.185426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b452068-e533-4b6c-b37b-db87c13ad7c2 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:10.319283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd0eb-2d34-4fe0-9beb-46637fef8bd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:10.320271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd0eb-2d34-4fe0-9beb-46637fef8bd9 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:10.320678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd0eb-2d34-4fe0-9beb-46637fef8bd9 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:10.321099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd0eb-2d34-4fe0-9beb-46637fef8bd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:10.323532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-eeefd0eb-2d34-4fe0-9beb-46637fef8bd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:10.388372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:10.429687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd0eb-2d34-4fe0-9beb-46637fef8bd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:10.431307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd0eb-2d34-4fe0-9beb-46637fef8bd9 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:10.454640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 690/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:42:12.640111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8976e5-05e9-403b-a3ce-0a5957b23962 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:12.640757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8976e5-05e9-403b-a3ce-0a5957b23962 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:12.640757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8976e5-05e9-403b-a3ce-0a5957b23962 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:12.642050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 696/1386] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:12.647393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3426a-0bd7-4a17-a45f-4d472a070d7e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:12.647865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3426a-0bd7-4a17-a45f-4d472a070d7e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:12.647973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3426a-0bd7-4a17-a45f-4d472a070d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:12.715919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b8f3426a-0bd7-4a17-a45f-4d472a070d7e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:12.717148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b8f3426a-0bd7-4a17-a45f-4d472a070d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:12.818890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 691/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:13.109073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8976e5-05e9-403b-a3ce-0a5957b23962 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:13.110723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7f8976e5-05e9-403b-a3ce-0a5957b23962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:13.187934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:13.257709 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8976e5-05e9-403b-a3ce-0a5957b23962 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:13.369034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0a765-2679-46e4-a18d-8b3b5f11fd9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:13.369781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a765-2679-46e4-a18d-8b3b5f11fd9d service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:13.370058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a765-2679-46e4-a18d-8b3b5f11fd9d service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:13.370324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a765-2679-46e4-a18d-8b3b5f11fd9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:13.371950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-51c0a765-2679-46e4-a18d-8b3b5f11fd9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:13.409948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:13.468253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0a765-2679-46e4-a18d-8b3b5f11fd9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:13.469905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0a765-2679-46e4-a18d-8b3b5f11fd9d service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:13.489195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 697/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:14.070557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a967b0-cdc6-4d0b-9fbf-db5fe01b78a3 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:14.071121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a967b0-cdc6-4d0b-9fbf-db5fe01b78a3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:14.071121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a967b0-cdc6-4d0b-9fbf-db5fe01b78a3 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:14.072450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 692/1389] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:14.077635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89033b7-41eb-48f2-abce-94cc5267724a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:14.078064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89033b7-41eb-48f2-abce-94cc5267724a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:14.078212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89033b7-41eb-48f2-abce-94cc5267724a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:14.142878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d89033b7-41eb-48f2-abce-94cc5267724a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:14.144083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d89033b7-41eb-48f2-abce-94cc5267724a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:14.243474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 698/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:14.541960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b430f-77ec-46d0-9a1f-f3e995a03e14 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:14.542250 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b430f-77ec-46d0-9a1f-f3e995a03e14 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:14.542527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b430f-77ec-46d0-9a1f-f3e995a03e14 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:14.543878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 693/1391] 10.222.0.23 () {54 vars in 966 bytes} [Tue Sep 13 19:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:14.549128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc4e7-820b-4361-a635-16cce57b3cec None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:14.549512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc4e7-820b-4361-a635-16cce57b3cec None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:14.549886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6adc4e7-820b-4361-a635-16cce57b3cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:14.566210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:14.628103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-c6adc4e7-820b-4361-a635-16cce57b3cec None None] MFA Rules not processed for user `17d78c1e1b6e46e3ba6271e6743d3c73`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:14.629675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c6adc4e7-820b-4361-a635-16cce57b3cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:14.704661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 699/1392] 10.222.0.23 () {58 vars in 1063 bytes} [Tue Sep 13 19:42:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:14.714360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b430f-77ec-46d0-9a1f-f3e995a03e14 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:14.715974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b73b430f-77ec-46d0-9a1f-f3e995a03e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:14.748108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:14.821569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b430f-77ec-46d0-9a1f-f3e995a03e14 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:14.895837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b02a6-87a8-49bd-a2f2-b12620652ccc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d78c1e1b6e46e3ba6271e6743d3c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:14.896738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b02a6-87a8-49bd-a2f2-b12620652ccc service glance] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:14.897016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b02a6-87a8-49bd-a2f2-b12620652ccc service glance] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:14.897287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b02a6-87a8-49bd-a2f2-b12620652ccc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:14.898929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-317b02a6-87a8-49bd-a2f2-b12620652ccc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:14.977601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:15.010495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b02a6-87a8-49bd-a2f2-b12620652ccc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:15.011961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b02a6-87a8-49bd-a2f2-b12620652ccc service glance] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:15.031952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 694/1393] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:15.700303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90dd4-fe79-4460-812e-de978c41dffe None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:15.700884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90dd4-fe79-4460-812e-de978c41dffe None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:15.701001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d90dd4-fe79-4460-812e-de978c41dffe None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:15.704085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 700/1394] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:15.711941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558f71-a4a1-4437-bcc1-58ced8179b1c None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:15.711941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558f71-a4a1-4437-bcc1-58ced8179b1c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:15.711941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558f71-a4a1-4437-bcc1-58ced8179b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:15.794541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-cf558f71-a4a1-4437-bcc1-58ced8179b1c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:15.795729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cf558f71-a4a1-4437-bcc1-58ced8179b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:15.797878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0375-198d-4bed-9f05-33f1a52cb63a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:15.798335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0375-198d-4bed-9f05-33f1a52cb63a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:15.798675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f0375-198d-4bed-9f05-33f1a52cb63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:15.869503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d07f0375-198d-4bed-9f05-33f1a52cb63a None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:15.870751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d07f0375-198d-4bed-9f05-33f1a52cb63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:15.894431 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 695/1395] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:15.925243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:15.956612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 701/1396] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:42:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:42:16.002651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951943e-00d3-4d91-9d81-4d45be205057 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:16.002936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951943e-00d3-4d91-9d81-4d45be205057 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:16.003315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951943e-00d3-4d91-9d81-4d45be205057 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:16.005303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 696/1397] 10.222.0.23 () {54 vars in 943 bytes} [Tue Sep 13 19:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:16.011967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ef3c9-6ae5-4abe-beec-c0e19f88ef3e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:16.012257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ef3c9-6ae5-4abe-beec-c0e19f88ef3e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:16.012531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ef3c9-6ae5-4abe-beec-c0e19f88ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:16.089026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-6b3ef3c9-6ae5-4abe-beec-c0e19f88ef3e None None] MFA Rules not processed for user `7d93aa681e2a4427a61fe0bf472c60ec`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:16.090181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6b3ef3c9-6ae5-4abe-beec-c0e19f88ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:16.181893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951943e-00d3-4d91-9d81-4d45be205057 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:16.183489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7951943e-00d3-4d91-9d81-4d45be205057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:16.197668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 702/1398] 10.222.0.23 () {58 vars in 1040 bytes} [Tue Sep 13 19:42:16 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:16.207483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ef3c9-6ae5-4abe-beec-c0e19f88ef3e None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:16.216997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ef3c9-6ae5-4abe-beec-c0e19f88ef3e None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:16.225886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:16.243441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:16.322576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f9e19-e4b2-448c-b97e-35230d4b4e98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:16.323184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f9e19-e4b2-448c-b97e-35230d4b4e98 service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:16.323458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f9e19-e4b2-448c-b97e-35230d4b4e98 service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:16.323726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f9e19-e4b2-448c-b97e-35230d4b4e98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:16.325353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-405f9e19-e4b2-448c-b97e-35230d4b4e98 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:16.335592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951943e-00d3-4d91-9d81-4d45be205057 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:16.410791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f9e19-e4b2-448c-b97e-35230d4b4e98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:16.412416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f9e19-e4b2-448c-b97e-35230d4b4e98 service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:16.420555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 703/1399] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 19:42:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:42:16.452125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:16.454799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de64f72-2c60-47a4-bfc5-1850e0458446 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:16.455574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64f72-2c60-47a4-bfc5-1850e0458446 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:16.456023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64f72-2c60-47a4-bfc5-1850e0458446 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:16.456499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de64f72-2c60-47a4-bfc5-1850e0458446 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:16.458857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8de64f72-2c60-47a4-bfc5-1850e0458446 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:16.562226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de64f72-2c60-47a4-bfc5-1850e0458446 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:16.563835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de64f72-2c60-47a4-bfc5-1850e0458446 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:16.589849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 697/1400] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:42:18.797835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e612f8-9f4a-47d1-9443-7c0880994c4c None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:18.798440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e612f8-9f4a-47d1-9443-7c0880994c4c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:18.798440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e612f8-9f4a-47d1-9443-7c0880994c4c None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:18.800881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 704/1401] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:18.806312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be36ed1-86b1-4359-980c-7baf945d725d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:18.806886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be36ed1-86b1-4359-980c-7baf945d725d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:18.806995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be36ed1-86b1-4359-980c-7baf945d725d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:18.887832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-6be36ed1-86b1-4359-980c-7baf945d725d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:18.889095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6be36ed1-86b1-4359-980c-7baf945d725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:18.978445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:19.024556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 698/1402] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:19.327404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e612f8-9f4a-47d1-9443-7c0880994c4c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:19.328918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f1e612f8-9f4a-47d1-9443-7c0880994c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:19.419476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:19.613480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e612f8-9f4a-47d1-9443-7c0880994c4c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:19.767312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6b129-fa8a-4e26-a636-7c827d371c4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:19.768259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6b129-fa8a-4e26-a636-7c827d371c4a service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:19.768863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6b129-fa8a-4e26-a636-7c827d371c4a service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:19.769248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6b129-fa8a-4e26-a636-7c827d371c4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:19.771611 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-cbc6b129-fa8a-4e26-a636-7c827d371c4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:19.792459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:19.884744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6b129-fa8a-4e26-a636-7c827d371c4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:19.886484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6b129-fa8a-4e26-a636-7c827d371c4a service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:19.913837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 705/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:22.380526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d0f9a-464a-42a0-9bb1-a02112797d42 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:22.381107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d0f9a-464a-42a0-9bb1-a02112797d42 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:22.381295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d0f9a-464a-42a0-9bb1-a02112797d42 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:22.383271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 699/1404] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:22.389626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1e7b0-74b7-4ef1-921f-82e1c4576b77 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:22.390210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1e7b0-74b7-4ef1-921f-82e1c4576b77 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:22.390210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1e7b0-74b7-4ef1-921f-82e1c4576b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:22.461475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-45b1e7b0-74b7-4ef1-921f-82e1c4576b77 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:22.462640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-45b1e7b0-74b7-4ef1-921f-82e1c4576b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:22.568992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:22.577038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 706/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:22.872546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d0f9a-464a-42a0-9bb1-a02112797d42 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:22.874096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-395d0f9a-464a-42a0-9bb1-a02112797d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:23.041129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d0f9a-464a-42a0-9bb1-a02112797d42 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:23.104138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:23.176088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7170c5-c5ce-4236-8cc8-6608c74c9b23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:23.176703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7170c5-c5ce-4236-8cc8-6608c74c9b23 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:23.176982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7170c5-c5ce-4236-8cc8-6608c74c9b23 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:23.177272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7170c5-c5ce-4236-8cc8-6608c74c9b23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:23.178882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ab7170c5-c5ce-4236-8cc8-6608c74c9b23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:23.283752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7170c5-c5ce-4236-8cc8-6608c74c9b23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:23.286037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7170c5-c5ce-4236-8cc8-6608c74c9b23 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:23.307384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 700/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:25.595686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e064-6563-42a0-8c5a-2f96fdf43cce None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:25.596665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e064-6563-42a0-8c5a-2f96fdf43cce None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:25.597222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71e064-6563-42a0-8c5a-2f96fdf43cce None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:25.599198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 707/1407] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:25.604255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e805f70-2fe8-4a37-94c6-94a0e5123d1d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:25.604732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e805f70-2fe8-4a37-94c6-94a0e5123d1d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:25.604825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e805f70-2fe8-4a37-94c6-94a0e5123d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:25.684117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1e805f70-2fe8-4a37-94c6-94a0e5123d1d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:25.685342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1e805f70-2fe8-4a37-94c6-94a0e5123d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:25.699081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:25.805760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 701/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:26.087725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71e064-6563-42a0-8c5a-2f96fdf43cce None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:26.089254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ba71e064-6563-42a0-8c5a-2f96fdf43cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:26.251614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71e064-6563-42a0-8c5a-2f96fdf43cce None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:26.328747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:26.381357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26244fd7-0644-49b1-a5a3-4ff0dbe67d3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:26.382025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244fd7-0644-49b1-a5a3-4ff0dbe67d3a service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:26.382308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244fd7-0644-49b1-a5a3-4ff0dbe67d3a service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:26.382578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26244fd7-0644-49b1-a5a3-4ff0dbe67d3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:26.384180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-26244fd7-0644-49b1-a5a3-4ff0dbe67d3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:26.489029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244fd7-0644-49b1-a5a3-4ff0dbe67d3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:26.490750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26244fd7-0644-49b1-a5a3-4ff0dbe67d3a service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:26.510996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 708/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:35.795924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=38 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:36.508421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=64 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:39.016557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec82d5-5e7d-43ce-a05f-ce775b0025d5 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:39.017046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec82d5-5e7d-43ce-a05f-ce775b0025d5 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:39.017160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec82d5-5e7d-43ce-a05f-ce775b0025d5 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:39.019235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 702/1410] 10.222.0.23 () {54 vars in 918 bytes} [Tue Sep 13 19:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:39.024443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed297-5a65-47e0-9e9d-466862c6cb9d None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:39.024887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed297-5a65-47e0-9e9d-466862c6cb9d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:39.025004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed297-5a65-47e0-9e9d-466862c6cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:39.099066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-599ed297-5a65-47e0-9e9d-466862c6cb9d None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:39.100295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-599ed297-5a65-47e0-9e9d-466862c6cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:39.207110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 709/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Sep 13 19:42:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:39.221730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e78c3e-bc98-42c7-b762-9952623248db None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:39.222014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e78c3e-bc98-42c7-b762-9952623248db None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:39.222288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e78c3e-bc98-42c7-b762-9952623248db None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:39.223609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 703/1412] 10.222.0.23 () {54 vars in 918 bytes} [Tue Sep 13 19:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:39.228529 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e9e0-6864-47f2-a67d-1ffbc3410c1a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:39.228812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e9e0-6864-47f2-a67d-1ffbc3410c1a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:39.229081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311e9e0-6864-47f2-a67d-1ffbc3410c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:39.300894 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:39.303778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-b311e9e0-6864-47f2-a67d-1ffbc3410c1a None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:39.305648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b311e9e0-6864-47f2-a67d-1ffbc3410c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:39.418598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 710/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Sep 13 19:42:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:41.967579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e34cdd-3523-40d8-ad4a-711cac566dc1 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:41.968050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e34cdd-3523-40d8-ad4a-711cac566dc1 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:41.968165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e34cdd-3523-40d8-ad4a-711cac566dc1 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:41.969475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:41.975181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918b55-995d-4fc2-8eb2-139dcb533442 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:41.975642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918b55-995d-4fc2-8eb2-139dcb533442 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:41.975747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5918b55-995d-4fc2-8eb2-139dcb533442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:42.042531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e5918b55-995d-4fc2-8eb2-139dcb533442 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:42.043722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e5918b55-995d-4fc2-8eb2-139dcb533442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:42.098317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:42.138729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:42.429283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e34cdd-3523-40d8-ad4a-711cac566dc1 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:42.430828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-44e34cdd-3523-40d8-ad4a-711cac566dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:42.574689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e34cdd-3523-40d8-ad4a-711cac566dc1 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:42.646895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:42.686587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a5a9d-970a-47a5-b24b-aa9d329e608b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:42.687123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5a9d-970a-47a5-b24b-aa9d329e608b service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:42.687399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5a9d-970a-47a5-b24b-aa9d329e608b service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:42.687666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5a9d-970a-47a5-b24b-aa9d329e608b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:42.689241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a16a5a9d-970a-47a5-b24b-aa9d329e608b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:42.777414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a5a9d-970a-47a5-b24b-aa9d329e608b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:42.779018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a5a9d-970a-47a5-b24b-aa9d329e608b service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:42.796984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 705/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:43.938863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77cef9-230e-4b9a-8dbc-273ecf873979 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:43.939449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77cef9-230e-4b9a-8dbc-273ecf873979 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:43.939449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77cef9-230e-4b9a-8dbc-273ecf873979 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:43.942095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:43.948399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fa22b-e8d3-460e-9d47-c4314f34c0d4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:43.948839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fa22b-e8d3-460e-9d47-c4314f34c0d4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:43.948963 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fa22b-e8d3-460e-9d47-c4314f34c0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:44.026677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-4b8fa22b-e8d3-460e-9d47-c4314f34c0d4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:44.027870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4b8fa22b-e8d3-460e-9d47-c4314f34c0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:44.054125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:44.121546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:44.272425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77cef9-230e-4b9a-8dbc-273ecf873979 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:44.273946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5d77cef9-230e-4b9a-8dbc-273ecf873979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:44.421572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77cef9-230e-4b9a-8dbc-273ecf873979 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:44.462162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:44.534879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b542-de25-4453-8322-2850d103c0fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:44.535405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b542-de25-4453-8322-2850d103c0fb service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:44.535681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b542-de25-4453-8322-2850d103c0fb service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:44.535950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b542-de25-4453-8322-2850d103c0fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:44.537547 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e301b542-de25-4453-8322-2850d103c0fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:44.626112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301b542-de25-4453-8322-2850d103c0fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:44.628611 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301b542-de25-4453-8322-2850d103c0fb service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:44.646952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 713/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:45.784362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52451519-8f30-4cd0-8902-d9b0535d341f None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:45.784973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52451519-8f30-4cd0-8902-d9b0535d341f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:45.784973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52451519-8f30-4cd0-8902-d9b0535d341f None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:45.786276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:45.792569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75b07-1c1d-46aa-a1f6-26528bee3010 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:45.792973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75b07-1c1d-46aa-a1f6-26528bee3010 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:45.793118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75b07-1c1d-46aa-a1f6-26528bee3010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:45.849098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:45.873126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fdd75b07-1c1d-46aa-a1f6-26528bee3010 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:45.874381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fdd75b07-1c1d-46aa-a1f6-26528bee3010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:45.968774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:46.118397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52451519-8f30-4cd0-8902-d9b0535d341f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:46.119895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-52451519-8f30-4cd0-8902-d9b0535d341f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:46.261842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52451519-8f30-4cd0-8902-d9b0535d341f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:46.274108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:46.374056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9b945-11f9-41a6-a1af-1d0c3eafb879 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:46.374580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9b945-11f9-41a6-a1af-1d0c3eafb879 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:46.374862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9b945-11f9-41a6-a1af-1d0c3eafb879 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:46.375135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9b945-11f9-41a6-a1af-1d0c3eafb879 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:46.376724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f3f9b945-11f9-41a6-a1af-1d0c3eafb879 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:46.467243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9b945-11f9-41a6-a1af-1d0c3eafb879 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:46.468879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9b945-11f9-41a6-a1af-1d0c3eafb879 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:46.491139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 708/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:47.593938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31569bd1-7d7b-4258-a4ab-1a68107875e7 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:47.594484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31569bd1-7d7b-4258-a4ab-1a68107875e7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:47.594484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31569bd1-7d7b-4258-a4ab-1a68107875e7 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:47.595830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:47.602103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c200b13-0f2f-4d9b-9132-30e256c80888 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:47.602511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c200b13-0f2f-4d9b-9132-30e256c80888 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:47.602648 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c200b13-0f2f-4d9b-9132-30e256c80888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:47.622446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:47.678620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-6c200b13-0f2f-4d9b-9132-30e256c80888 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:47.679822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6c200b13-0f2f-4d9b-9132-30e256c80888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:47.773599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:47.923028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31569bd1-7d7b-4258-a4ab-1a68107875e7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:47.924608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-31569bd1-7d7b-4258-a4ab-1a68107875e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:48.035615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:48.071716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31569bd1-7d7b-4258-a4ab-1a68107875e7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:48.184323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbd05d-ad01-4ea0-afe5-a925877e95d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:48.184862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbd05d-ad01-4ea0-afe5-a925877e95d0 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:48.185143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbd05d-ad01-4ea0-afe5-a925877e95d0 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:48.185699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbd05d-ad01-4ea0-afe5-a925877e95d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:48.187305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f7dbd05d-ad01-4ea0-afe5-a925877e95d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:48.261886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:48.275650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbd05d-ad01-4ea0-afe5-a925877e95d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:48.277239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbd05d-ad01-4ea0-afe5-a925877e95d0 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:48.295392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 716/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:49.414116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd182855-d9ba-425e-b2be-ef180f72518f None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:49.414689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd182855-d9ba-425e-b2be-ef180f72518f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:49.414689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd182855-d9ba-425e-b2be-ef180f72518f None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:49.416056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:49.422182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be5e26-3f67-41b1-b0b5-3ab9e0e6e8c7 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:49.422644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be5e26-3f67-41b1-b0b5-3ab9e0e6e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:49.422804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be5e26-3f67-41b1-b0b5-3ab9e0e6e8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:49.617217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-b0be5e26-3f67-41b1-b0b5-3ab9e0e6e8c7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:49.618488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b0be5e26-3f67-41b1-b0b5-3ab9e0e6e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:49.714376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:49.863611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd182855-d9ba-425e-b2be-ef180f72518f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:49.865138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bd182855-d9ba-425e-b2be-ef180f72518f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:49.953432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:50.016819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd182855-d9ba-425e-b2be-ef180f72518f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:50.129525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a113082d-bc25-4e28-87f4-d2faf4d8d4fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:50.130173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113082d-bc25-4e28-87f4-d2faf4d8d4fb service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:50.130454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113082d-bc25-4e28-87f4-d2faf4d8d4fb service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:50.130722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113082d-bc25-4e28-87f4-d2faf4d8d4fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:50.132312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a113082d-bc25-4e28-87f4-d2faf4d8d4fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:50.179840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:50.221134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113082d-bc25-4e28-87f4-d2faf4d8d4fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:50.222744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a113082d-bc25-4e28-87f4-d2faf4d8d4fb service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:50.240645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 711/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:51.306208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8619f34-28a5-4e3c-b238-04933fdd66f7 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:51.306836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8619f34-28a5-4e3c-b238-04933fdd66f7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:51.306836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8619f34-28a5-4e3c-b238-04933fdd66f7 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:51.308103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:51.315111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9492bf-3061-4b8d-9971-9ef251c9c2d3 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:51.315579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9492bf-3061-4b8d-9971-9ef251c9c2d3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:51.315690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9492bf-3061-4b8d-9971-9ef251c9c2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:51.390501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-0c9492bf-3061-4b8d-9971-9ef251c9c2d3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:51.391692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0c9492bf-3061-4b8d-9971-9ef251c9c2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:51.485831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:51.638203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8619f34-28a5-4e3c-b238-04933fdd66f7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:51.639762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d8619f34-28a5-4e3c-b238-04933fdd66f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:51.690562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:51.784198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8619f34-28a5-4e3c-b238-04933fdd66f7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:51.894027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fb64c-0313-4b3f-8ac4-77fca4c686ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:51.894552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fb64c-0313-4b3f-8ac4-77fca4c686ed service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:51.894826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fb64c-0313-4b3f-8ac4-77fca4c686ed service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:51.895096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fb64c-0313-4b3f-8ac4-77fca4c686ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:51.896694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e60fb64c-0313-4b3f-8ac4-77fca4c686ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:51.903433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:51.997962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fb64c-0313-4b3f-8ac4-77fca4c686ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:52.000268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fb64c-0313-4b3f-8ac4-77fca4c686ed service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:52.018490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 719/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:53.082307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694a5dd-1c2d-4e4d-bf21-565b68d14532 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:53.082858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694a5dd-1c2d-4e4d-bf21-565b68d14532 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:53.082858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694a5dd-1c2d-4e4d-bf21-565b68d14532 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:53.084175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:53.090704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79ec0-22f0-43b5-9933-60cc9a174edb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:53.091193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79ec0-22f0-43b5-9933-60cc9a174edb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:53.091304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f79ec0-22f0-43b5-9933-60cc9a174edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:53.167332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-42f79ec0-22f0-43b5-9933-60cc9a174edb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:53.168548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-42f79ec0-22f0-43b5-9933-60cc9a174edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:53.234998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:53.263880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:53.414135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694a5dd-1c2d-4e4d-bf21-565b68d14532 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:53.415642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4694a5dd-1c2d-4e4d-bf21-565b68d14532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:53.433023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:53.558990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694a5dd-1c2d-4e4d-bf21-565b68d14532 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:53.646748 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:53.671909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d17cea-c802-4650-bf23-d1b9487c259a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:53.672523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d17cea-c802-4650-bf23-d1b9487c259a service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:53.672806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d17cea-c802-4650-bf23-d1b9487c259a service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:53.673078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d17cea-c802-4650-bf23-d1b9487c259a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:53.675205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d6d17cea-c802-4650-bf23-d1b9487c259a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:53.764225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d17cea-c802-4650-bf23-d1b9487c259a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:53.765857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d17cea-c802-4650-bf23-d1b9487c259a service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:53.783676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 714/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:54.833776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b12ab5-769f-4790-b7d0-59a5ec3d08f8 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:54.834302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b12ab5-769f-4790-b7d0-59a5ec3d08f8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:54.834302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b12ab5-769f-4790-b7d0-59a5ec3d08f8 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:54.835662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:54.841832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcf6e9-08b8-4d43-a7c5-0b70cdfa98d5 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:54.842271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcf6e9-08b8-4d43-a7c5-0b70cdfa98d5 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:54.842420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fcf6e9-08b8-4d43-a7c5-0b70cdfa98d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:54.916651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b6fcf6e9-08b8-4d43-a7c5-0b70cdfa98d5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:54.917804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b6fcf6e9-08b8-4d43-a7c5-0b70cdfa98d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:54.959227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:55.013794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:55.167763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b12ab5-769f-4790-b7d0-59a5ec3d08f8 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:55.169205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f6b12ab5-769f-4790-b7d0-59a5ec3d08f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:55.316989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b12ab5-769f-4790-b7d0-59a5ec3d08f8 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:55.375436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:55.432805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112c3306-644e-415d-a8e4-c40c4f1bddac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:55.433342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c3306-644e-415d-a8e4-c40c4f1bddac service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:55.433631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c3306-644e-415d-a8e4-c40c4f1bddac service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:55.433911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112c3306-644e-415d-a8e4-c40c4f1bddac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:55.435757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-112c3306-644e-415d-a8e4-c40c4f1bddac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:55.523952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c3306-644e-415d-a8e4-c40c4f1bddac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:55.525510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112c3306-644e-415d-a8e4-c40c4f1bddac service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:55.543300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 722/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:56.601832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920eb3-cc7a-47a1-8861-c11ce731e48d None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:56.602382 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920eb3-cc7a-47a1-8861-c11ce731e48d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:56.602382 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920eb3-cc7a-47a1-8861-c11ce731e48d None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:56.603731 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:56.610010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f37f6-7e5a-420e-bad6-1aa90f9c29ff None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:56.610464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f37f6-7e5a-420e-bad6-1aa90f9c29ff None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:56.610571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f37f6-7e5a-420e-bad6-1aa90f9c29ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:56.685507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-7c2f37f6-7e5a-420e-bad6-1aa90f9c29ff None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:56.686738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7c2f37f6-7e5a-420e-bad6-1aa90f9c29ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:56.688732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:56.790338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:56.940622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1920eb3-cc7a-47a1-8861-c11ce731e48d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:56.942150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f1920eb3-cc7a-47a1-8861-c11ce731e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:57.089467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1920eb3-cc7a-47a1-8861-c11ce731e48d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:57.116607 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:57.200617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc0acf-3ab1-4541-a2fc-8065289ed149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:57.201140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0acf-3ab1-4541-a2fc-8065289ed149 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:57.201416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0acf-3ab1-4541-a2fc-8065289ed149 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:57.201714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc0acf-3ab1-4541-a2fc-8065289ed149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:57.203291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5cbc0acf-3ab1-4541-a2fc-8065289ed149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:57.291978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc0acf-3ab1-4541-a2fc-8065289ed149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:57.293595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbc0acf-3ab1-4541-a2fc-8065289ed149 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:57.312252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 717/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:59.365577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aaabb-0c31-4f03-8813-e39c13e409e7 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:59.366129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aaabb-0c31-4f03-8813-e39c13e409e7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:59.366129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aaabb-0c31-4f03-8813-e39c13e409e7 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:59.367528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:59.372849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff54b3-a86b-4a39-9f66-a15b47e0257d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:59.373301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff54b3-a86b-4a39-9f66-a15b47e0257d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:59.373399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff54b3-a86b-4a39-9f66-a15b47e0257d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:59.416286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:59.442572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-beff54b3-a86b-4a39-9f66-a15b47e0257d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:59.443773 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-beff54b3-a86b-4a39-9f66-a15b47e0257d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:59.538558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:59.827792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5aaabb-0c31-4f03-8813-e39c13e409e7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:59.829314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0b5aaabb-0c31-4f03-8813-e39c13e409e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:59.935972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5aaabb-0c31-4f03-8813-e39c13e409e7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:59.968291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:00.013967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff0e27-c6cd-473b-9293-9143427a2c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:00.014717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0e27-c6cd-473b-9293-9143427a2c63 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:00.015000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0e27-c6cd-473b-9293-9143427a2c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:00.015270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0e27-c6cd-473b-9293-9143427a2c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:00.016882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-abff0e27-c6cd-473b-9293-9143427a2c63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:00.125551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff0e27-c6cd-473b-9293-9143427a2c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:00.127537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff0e27-c6cd-473b-9293-9143427a2c63 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:00.146494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 725/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:02.487564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce165-4709-491e-a9ff-233bebd917af None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:02.488149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce165-4709-491e-a9ff-233bebd917af None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:02.488149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce165-4709-491e-a9ff-233bebd917af None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:02.489430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:02.494712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e861736-b0eb-43b0-a2e8-ccee085e174e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:02.495140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e861736-b0eb-43b0-a2e8-ccee085e174e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:02.495281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e861736-b0eb-43b0-a2e8-ccee085e174e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:02.559169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9e861736-b0eb-43b0-a2e8-ccee085e174e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:02.560535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9e861736-b0eb-43b0-a2e8-ccee085e174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:02.573339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:02.656033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:02.937278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce165-4709-491e-a9ff-233bebd917af None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:02.939635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-845ce165-4709-491e-a9ff-233bebd917af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:03.042400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce165-4709-491e-a9ff-233bebd917af None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:03.055941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:03.118884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787905c-93f5-48d6-8b19-0957125df7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:03.119485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787905c-93f5-48d6-8b19-0957125df7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:03.119768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787905c-93f5-48d6-8b19-0957125df7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:03.120042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787905c-93f5-48d6-8b19-0957125df7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:03.121650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2787905c-93f5-48d6-8b19-0957125df7d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:03.230316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787905c-93f5-48d6-8b19-0957125df7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:03.231683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787905c-93f5-48d6-8b19-0957125df7d6 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:03.250318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 720/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:05.724223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08022b-0b27-4947-8e26-31fa89604bd7 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:05.724810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08022b-0b27-4947-8e26-31fa89604bd7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:05.724810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08022b-0b27-4947-8e26-31fa89604bd7 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:05.726146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:05.731502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e2a3-5b15-4ce1-b8d4-39f28e0643a3 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:05.731893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e2a3-5b15-4ce1-b8d4-39f28e0643a3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:05.732047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7e2a3-5b15-4ce1-b8d4-39f28e0643a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:05.779855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:05.798772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-6dc7e2a3-5b15-4ce1-b8d4-39f28e0643a3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:05.799970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6dc7e2a3-5b15-4ce1-b8d4-39f28e0643a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:05.896485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:06.188571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08022b-0b27-4947-8e26-31fa89604bd7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:06.190143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9e08022b-0b27-4947-8e26-31fa89604bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:06.292162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08022b-0b27-4947-8e26-31fa89604bd7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:06.331823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:06.365845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db7f2e-dcac-487c-b3e8-663c7ff910aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:06.366449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7f2e-dcac-487c-b3e8-663c7ff910aa service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:06.366724 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7f2e-dcac-487c-b3e8-663c7ff910aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:06.366991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db7f2e-dcac-487c-b3e8-663c7ff910aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:06.368566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-91db7f2e-dcac-487c-b3e8-663c7ff910aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:06.476486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db7f2e-dcac-487c-b3e8-663c7ff910aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:06.477989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db7f2e-dcac-487c-b3e8-663c7ff910aa service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:06.495599 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 728/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:08.772852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0ea60-a056-48c6-bf5e-288a6f34cc3b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:08.773432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0ea60-a056-48c6-bf5e-288a6f34cc3b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:08.773432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de0ea60-a056-48c6-bf5e-288a6f34cc3b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:08.774768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:08.780031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477177a-919a-43e4-b12b-b6f9a2b9d465 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:08.780427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477177a-919a-43e4-b12b-b6f9a2b9d465 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:08.780598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477177a-919a-43e4-b12b-b6f9a2b9d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:08.846254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9477177a-919a-43e4-b12b-b6f9a2b9d465 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:08.847473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9477177a-919a-43e4-b12b-b6f9a2b9d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:08.868111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:08.945786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:09.230409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0ea60-a056-48c6-bf5e-288a6f34cc3b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:09.231964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5de0ea60-a056-48c6-bf5e-288a6f34cc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:09.333220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de0ea60-a056-48c6-bf5e-288a6f34cc3b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:09.353860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:09.407804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfafe6-8b59-43bf-a374-8deef605ab83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:09.408423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfafe6-8b59-43bf-a374-8deef605ab83 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:09.408706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfafe6-8b59-43bf-a374-8deef605ab83 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:09.408980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfafe6-8b59-43bf-a374-8deef605ab83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:09.410622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-61bfafe6-8b59-43bf-a374-8deef605ab83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:09.518605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bfafe6-8b59-43bf-a374-8deef605ab83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:09.519966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bfafe6-8b59-43bf-a374-8deef605ab83 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:09.537921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 723/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:11.761439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600abb-86e6-4810-aab4-457b9f1b1736 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:11.762071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600abb-86e6-4810-aab4-457b9f1b1736 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:11.762071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600abb-86e6-4810-aab4-457b9f1b1736 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:11.763378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:11.769283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9094b1-0aad-454d-8298-158e86588840 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:11.769714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9094b1-0aad-454d-8298-158e86588840 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:11.769907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9094b1-0aad-454d-8298-158e86588840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:11.832394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:11.835087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-4c9094b1-0aad-454d-8298-158e86588840 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:11.836907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4c9094b1-0aad-454d-8298-158e86588840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:11.932961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:12.218068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600abb-86e6-4810-aab4-457b9f1b1736 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:12.219668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-73600abb-86e6-4810-aab4-457b9f1b1736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:12.323368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600abb-86e6-4810-aab4-457b9f1b1736 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:12.372248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:12.396606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f211092-adb4-4926-a4bb-f3e90ba95718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:12.397208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211092-adb4-4926-a4bb-f3e90ba95718 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:12.397488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211092-adb4-4926-a4bb-f3e90ba95718 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:12.397798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f211092-adb4-4926-a4bb-f3e90ba95718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:12.399400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2f211092-adb4-4926-a4bb-f3e90ba95718 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:12.507652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f211092-adb4-4926-a4bb-f3e90ba95718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:12.509227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f211092-adb4-4926-a4bb-f3e90ba95718 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:12.527662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 731/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:14.739266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af495d-dc81-4d83-983a-693573595a6c None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:14.739826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af495d-dc81-4d83-983a-693573595a6c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:14.739826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af495d-dc81-4d83-983a-693573595a6c None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:14.741153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:14.746491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411df1b-c388-4a08-b1e4-a4613c345b7c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:14.746929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411df1b-c388-4a08-b1e4-a4613c345b7c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:14.747038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411df1b-c388-4a08-b1e4-a4613c345b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:14.811617 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4411df1b-c388-4a08-b1e4-a4613c345b7c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:14.812823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4411df1b-c388-4a08-b1e4-a4613c345b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:14.841102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:14.909482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:15.192770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af495d-dc81-4d83-983a-693573595a6c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:15.194403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-00af495d-dc81-4d83-983a-693573595a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:15.299447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af495d-dc81-4d83-983a-693573595a6c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:15.327714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:15.374441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e243798-010f-4f48-8ed6-c76dbad198f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:15.375085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e243798-010f-4f48-8ed6-c76dbad198f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:15.375473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e243798-010f-4f48-8ed6-c76dbad198f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:15.376295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e243798-010f-4f48-8ed6-c76dbad198f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:15.377919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3e243798-010f-4f48-8ed6-c76dbad198f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:15.487601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e243798-010f-4f48-8ed6-c76dbad198f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:15.489068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e243798-010f-4f48-8ed6-c76dbad198f5 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:15.506557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 726/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:17.691892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09eb81a-ad81-4556-acb9-a9b559e837a0 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:17.692470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09eb81a-ad81-4556-acb9-a9b559e837a0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:17.692470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09eb81a-ad81-4556-acb9-a9b559e837a0 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:17.693821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:17.699518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bfafc-26a1-4109-bea8-a875178f5db8 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:17.699928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bfafc-26a1-4109-bea8-a875178f5db8 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:17.700065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1bfafc-26a1-4109-bea8-a875178f5db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:17.764443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ff1bfafc-26a1-4109-bea8-a875178f5db8 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:17.765664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ff1bfafc-26a1-4109-bea8-a875178f5db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:17.773832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:17.861215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:18.143644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09eb81a-ad81-4556-acb9-a9b559e837a0 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:18.145229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f09eb81a-ad81-4556-acb9-a9b559e837a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:18.248062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09eb81a-ad81-4556-acb9-a9b559e837a0 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:18.304745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:18.323376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408d5f6-29eb-49bc-8108-3fef73138d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:18.323974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408d5f6-29eb-49bc-8108-3fef73138d7a service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.324256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408d5f6-29eb-49bc-8108-3fef73138d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.324519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408d5f6-29eb-49bc-8108-3fef73138d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.326157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7408d5f6-29eb-49bc-8108-3fef73138d7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:18.435358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408d5f6-29eb-49bc-8108-3fef73138d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:18.436771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408d5f6-29eb-49bc-8108-3fef73138d7a service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:18.454631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 734/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:20.649223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54047411-b3b2-4350-bc7b-57151365019c None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:20.649841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54047411-b3b2-4350-bc7b-57151365019c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:20.649841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54047411-b3b2-4350-bc7b-57151365019c None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.651145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:20.656472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9256e-8114-4984-86d4-729919642878 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:20.656920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9256e-8114-4984-86d4-729919642878 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:20.657036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9256e-8114-4984-86d4-729919642878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.723363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-83a9256e-8114-4984-86d4-729919642878 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:20.724584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-83a9256e-8114-4984-86d4-729919642878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:20.761463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:20.821314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:21.104720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54047411-b3b2-4350-bc7b-57151365019c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:21.106283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-54047411-b3b2-4350-bc7b-57151365019c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:21.211223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54047411-b3b2-4350-bc7b-57151365019c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:21.250883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:21.286922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e544a-196c-4402-81c5-35ff57678a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:21.287467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e544a-196c-4402-81c5-35ff57678a24 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:21.287751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e544a-196c-4402-81c5-35ff57678a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:21.288027 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e544a-196c-4402-81c5-35ff57678a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:21.289642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-610e544a-196c-4402-81c5-35ff57678a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:21.404102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e544a-196c-4402-81c5-35ff57678a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:21.405587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e544a-196c-4402-81c5-35ff57678a24 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:21.424259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 729/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:23.612952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4738b-1960-47ac-af83-992d52057c39 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.613554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4738b-1960-47ac-af83-992d52057c39 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.613554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb4738b-1960-47ac-af83-992d52057c39 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.614899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:23.620502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683796-aadc-49b9-bb2b-fa8ab34f602d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.620915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683796-aadc-49b9-bb2b-fa8ab34f602d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.621119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683796-aadc-49b9-bb2b-fa8ab34f602d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.685935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e2683796-aadc-49b9-bb2b-fa8ab34f602d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:23.687148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e2683796-aadc-49b9-bb2b-fa8ab34f602d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.705289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.784756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:24.071078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4738b-1960-47ac-af83-992d52057c39 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:24.072626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2eb4738b-1960-47ac-af83-992d52057c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:24.176205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb4738b-1960-47ac-af83-992d52057c39 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:24.241875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:24.249023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbcdd0-2220-46f2-8a9a-4f77c9526ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:24.249647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbcdd0-2220-46f2-8a9a-4f77c9526ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:24.249931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbcdd0-2220-46f2-8a9a-4f77c9526ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:24.250199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbcdd0-2220-46f2-8a9a-4f77c9526ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:24.251785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9dfbcdd0-2220-46f2-8a9a-4f77c9526ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:24.364309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfbcdd0-2220-46f2-8a9a-4f77c9526ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:24.365789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfbcdd0-2220-46f2-8a9a-4f77c9526ff6 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:24.383302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 737/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:26.547319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55215-4d28-41b5-a8b4-690d1684c671 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:26.547993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55215-4d28-41b5-a8b4-690d1684c671 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:26.547993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae55215-4d28-41b5-a8b4-690d1684c671 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:26.549199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:26.554612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65eb0-f629-45d8-b165-c7531b814c0a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:26.555029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65eb0-f629-45d8-b165-c7531b814c0a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:26.555141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b65eb0-f629-45d8-b165-c7531b814c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:26.620973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d5b65eb0-f629-45d8-b165-c7531b814c0a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:26.622201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d5b65eb0-f629-45d8-b165-c7531b814c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:26.670659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:26.720049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:27.003734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae55215-4d28-41b5-a8b4-690d1684c671 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:27.005296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-aae55215-4d28-41b5-a8b4-690d1684c671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:27.108955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae55215-4d28-41b5-a8b4-690d1684c671 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:27.157588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:27.182538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4475dc1-cb42-4446-b569-5658927849cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:27.183147 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4475dc1-cb42-4446-b569-5658927849cd service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:27.183421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4475dc1-cb42-4446-b569-5658927849cd service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:27.183686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4475dc1-cb42-4446-b569-5658927849cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:27.185519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f4475dc1-cb42-4446-b569-5658927849cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:27.292354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4475dc1-cb42-4446-b569-5658927849cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:27.293756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4475dc1-cb42-4446-b569-5658927849cd service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:27.311893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 732/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:36.718484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=22 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:37.309718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=66 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.316493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d393142-5067-4ae2-bdef-8a6e7b1c51ee None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.316992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d393142-5067-4ae2-bdef-8a6e7b1c51ee None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.317109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d393142-5067-4ae2-bdef-8a6e7b1c51ee None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.318431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 739/1471] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:47.323638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5477c-65e9-4e67-b507-5bcbdcb81b2a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.324088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5477c-65e9-4e67-b507-5bcbdcb81b2a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.324320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5477c-65e9-4e67-b507-5bcbdcb81b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.390720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-efa5477c-65e9-4e67-b507-5bcbdcb81b2a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:47.391927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-efa5477c-65e9-4e67-b507-5bcbdcb81b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:47.487558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 733/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:47.788973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232d3bf-24cb-4afe-b934-55a0a3f9cba8 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.789272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232d3bf-24cb-4afe-b934-55a0a3f9cba8 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.789542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0232d3bf-24cb-4afe-b934-55a0a3f9cba8 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.790940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 740/1473] 10.222.0.23 () {54 vars in 966 bytes} [Tue Sep 13 19:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:47.795774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb14a6-bd3c-4e4a-8451-c5bb4b6c463b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.796115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb14a6-bd3c-4e4a-8451-c5bb4b6c463b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.796390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb14a6-bd3c-4e4a-8451-c5bb4b6c463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.873142 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3dfb14a6-bd3c-4e4a-8451-c5bb4b6c463b None None] MFA Rules not processed for user `17d78c1e1b6e46e3ba6271e6743d3c73`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:47.874387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3dfb14a6-bd3c-4e4a-8451-c5bb4b6c463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:47.895144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.950012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 734/1474] 10.222.0.23 () {58 vars in 1063 bytes} [Tue Sep 13 19:43:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:47.958276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232d3bf-24cb-4afe-b934-55a0a3f9cba8 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.959786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0232d3bf-24cb-4afe-b934-55a0a3f9cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:48.061995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0232d3bf-24cb-4afe-b934-55a0a3f9cba8 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:48.134054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4156999-4e1e-4f8b-a36b-43a13afc197f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d78c1e1b6e46e3ba6271e6743d3c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:48.134660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4156999-4e1e-4f8b-a36b-43a13afc197f service glance] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.134935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4156999-4e1e-4f8b-a36b-43a13afc197f service glance] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.135203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4156999-4e1e-4f8b-a36b-43a13afc197f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.136807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a4156999-4e1e-4f8b-a36b-43a13afc197f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:48.189558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.243353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4156999-4e1e-4f8b-a36b-43a13afc197f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:48.244772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4156999-4e1e-4f8b-a36b-43a13afc197f service glance] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:48.263380 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 741/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:50.497498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cce00d-40ae-46eb-92cc-2d154d49dad6 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:50.498104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cce00d-40ae-46eb-92cc-2d154d49dad6 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:50.498104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cce00d-40ae-46eb-92cc-2d154d49dad6 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:50.499554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 735/1476] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:50.505068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1e4f-c708-4328-b9d0-666f12c9bbb0 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:50.505516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1e4f-c708-4328-b9d0-666f12c9bbb0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:50.505666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1e4f-c708-4328-b9d0-666f12c9bbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:50.572211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-dc1a1e4f-c708-4328-b9d0-666f12c9bbb0 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:50.573412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-dc1a1e4f-c708-4328-b9d0-666f12c9bbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:50.677539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 742/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:50.951964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cce00d-40ae-46eb-92cc-2d154d49dad6 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:50.953481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-08cce00d-40ae-46eb-92cc-2d154d49dad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:51.077149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cce00d-40ae-46eb-92cc-2d154d49dad6 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:51.126986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:51.171894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2838dcb5-77d1-49e6-94e1-9cc5075bc0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:51.172447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838dcb5-77d1-49e6-94e1-9cc5075bc0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:51.172792 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838dcb5-77d1-49e6-94e1-9cc5075bc0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:51.173079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2838dcb5-77d1-49e6-94e1-9cc5075bc0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:51.173820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838dcb5-77d1-49e6-94e1-9cc5075bc0e4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:51.175718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING py.warnings [None req-2838dcb5-77d1-49e6-94e1-9cc5075bc0e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:43:51.175718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: warnings.warn(msg) Sep 13 19:43:51.175718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: Sep 13 19:43:51.176981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2838dcb5-77d1-49e6-94e1-9cc5075bc0e4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:51.196274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 736/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Sep 13 19:43:50 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:43:53.375092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b542f-744a-498e-8896-dcfb8b741f15 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.375692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b542f-744a-498e-8896-dcfb8b741f15 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.375692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b542f-744a-498e-8896-dcfb8b741f15 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.376974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 743/1479] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:53.382173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fe361-0317-4036-98dd-da8a5c33af5a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.382629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fe361-0317-4036-98dd-da8a5c33af5a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.382725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fe361-0317-4036-98dd-da8a5c33af5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.448601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-343fe361-0317-4036-98dd-da8a5c33af5a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:53.449826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-343fe361-0317-4036-98dd-da8a5c33af5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:53.546690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 737/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:53.839340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b542f-744a-498e-8896-dcfb8b741f15 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:53.840926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-949b542f-744a-498e-8896-dcfb8b741f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:53.851295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:53.992466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b542f-744a-498e-8896-dcfb8b741f15 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:54.070795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:54.102871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1dcf2-c0df-457a-87b0-added8f980fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:54.103404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dcf2-c0df-457a-87b0-added8f980fa service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:54.103678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dcf2-c0df-457a-87b0-added8f980fa service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:54.103945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dcf2-c0df-457a-87b0-added8f980fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:54.105512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-cff1dcf2-c0df-457a-87b0-added8f980fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:54.195659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1dcf2-c0df-457a-87b0-added8f980fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:54.197247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1dcf2-c0df-457a-87b0-added8f980fa service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:54.214728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 744/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:56.504053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bb5f6-af17-4828-a57e-75b5a15d8d1d None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.504601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bb5f6-af17-4828-a57e-75b5a15d8d1d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.504601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bb5f6-af17-4828-a57e-75b5a15d8d1d None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.506016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 738/1482] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:56.511794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126afae-eefc-4f08-a02d-a328aa7197de None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.512280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126afae-eefc-4f08-a02d-a328aa7197de None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.512387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126afae-eefc-4f08-a02d-a328aa7197de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.581657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f126afae-eefc-4f08-a02d-a328aa7197de None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:56.583160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f126afae-eefc-4f08-a02d-a328aa7197de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:56.629292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:56.681361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 745/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:56.970561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bb5f6-af17-4828-a57e-75b5a15d8d1d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:56.972102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-294bb5f6-af17-4828-a57e-75b5a15d8d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:56.995276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.076969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bb5f6-af17-4828-a57e-75b5a15d8d1d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.150626 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad70df6-449d-4d2d-a1bc-8daaed760408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.151237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad70df6-449d-4d2d-a1bc-8daaed760408 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.151513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad70df6-449d-4d2d-a1bc-8daaed760408 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.151781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad70df6-449d-4d2d-a1bc-8daaed760408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.153364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3ad70df6-449d-4d2d-a1bc-8daaed760408 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:57.344967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.379952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad70df6-449d-4d2d-a1bc-8daaed760408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:57.381384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad70df6-449d-4d2d-a1bc-8daaed760408 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:57.399267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 739/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:59.592610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b28c7f-b2ca-4c55-b80b-08f8e6697ba0 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:59.593195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b28c7f-b2ca-4c55-b80b-08f8e6697ba0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:59.593195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b28c7f-b2ca-4c55-b80b-08f8e6697ba0 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:59.594571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 746/1485] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:59.599884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aba80-730b-4d9a-b478-ae23c2963650 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:59.600306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aba80-730b-4d9a-b478-ae23c2963650 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:59.600417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2aba80-730b-4d9a-b478-ae23c2963650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:59.665201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-9d2aba80-730b-4d9a-b478-ae23c2963650 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:59.666467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9d2aba80-730b-4d9a-b478-ae23c2963650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:59.761540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 740/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:00.049806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b28c7f-b2ca-4c55-b80b-08f8e6697ba0 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:00.051374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-47b28c7f-b2ca-4c55-b80b-08f8e6697ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:00.153220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b28c7f-b2ca-4c55-b80b-08f8e6697ba0 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:00.233572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:00.235003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954a37c-877c-49c4-b3f7-ae996e9bee24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:00.235624 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a37c-877c-49c4-b3f7-ae996e9bee24 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:00.235971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a37c-877c-49c4-b3f7-ae996e9bee24 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:00.236490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a37c-877c-49c4-b3f7-ae996e9bee24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:00.238482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b954a37c-877c-49c4-b3f7-ae996e9bee24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:00.346459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954a37c-877c-49c4-b3f7-ae996e9bee24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:00.348016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954a37c-877c-49c4-b3f7-ae996e9bee24 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:00.365825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 747/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:02.637538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb45993-7017-4ede-801f-bc3d02a8d53b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.638135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb45993-7017-4ede-801f-bc3d02a8d53b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.638135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb45993-7017-4ede-801f-bc3d02a8d53b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.640010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 741/1488] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:44:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:02.645180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ff54-9122-4294-95db-6d4cc4b444d2 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.645572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ff54-9122-4294-95db-6d4cc4b444d2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.645889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88ff54-9122-4294-95db-6d4cc4b444d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.721859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ee88ff54-9122-4294-95db-6d4cc4b444d2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:02.723579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ee88ff54-9122-4294-95db-6d4cc4b444d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:02.783697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:02.827985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 748/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:03.118921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb45993-7017-4ede-801f-bc3d02a8d53b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:03.120464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-abb45993-7017-4ede-801f-bc3d02a8d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:03.158409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:03.226951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb45993-7017-4ede-801f-bc3d02a8d53b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:03.298709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b569825-62ae-4c88-bea0-f9d875da2316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:03.299231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b569825-62ae-4c88-bea0-f9d875da2316 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:03.299506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b569825-62ae-4c88-bea0-f9d875da2316 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:03.299771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b569825-62ae-4c88-bea0-f9d875da2316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:03.301342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2b569825-62ae-4c88-bea0-f9d875da2316 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:03.382931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:03.411276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b569825-62ae-4c88-bea0-f9d875da2316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:03.412729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b569825-62ae-4c88-bea0-f9d875da2316 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:03.431271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 742/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:05.789239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3f27-a0c3-4d30-aeae-ad7569d6042d None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.789845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3f27-a0c3-4d30-aeae-ad7569d6042d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.789845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3f27-a0c3-4d30-aeae-ad7569d6042d None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.791180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 749/1491] 10.222.0.23 () {54 vars in 927 bytes} [Tue Sep 13 19:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:05.796549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459ad9f-47bc-4c08-be3b-60108dfcea59 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.796967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459ad9f-47bc-4c08-be3b-60108dfcea59 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.797127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459ad9f-47bc-4c08-be3b-60108dfcea59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.861575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8459ad9f-47bc-4c08-be3b-60108dfcea59 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:05.862805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8459ad9f-47bc-4c08-be3b-60108dfcea59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:05.957995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 743/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Tue Sep 13 19:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:06.241812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3f27-a0c3-4d30-aeae-ad7569d6042d None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:06.243389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d1fd3f27-a0c3-4d30-aeae-ad7569d6042d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:06.344232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3f27-a0c3-4d30-aeae-ad7569d6042d None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:06.417035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6cdaa-988e-4ec7-a100-c675c27fc623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:06.417674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6cdaa-988e-4ec7-a100-c675c27fc623 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:06.417956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6cdaa-988e-4ec7-a100-c675c27fc623 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:06.418229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6cdaa-988e-4ec7-a100-c675c27fc623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:06.420395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-cac6cdaa-988e-4ec7-a100-c675c27fc623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:06.429013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:06.531655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6cdaa-988e-4ec7-a100-c675c27fc623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:06.533002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6cdaa-988e-4ec7-a100-c675c27fc623 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:06.550975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 750/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:15.956356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=82 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:16.548703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=50 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:25.785323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf9c3b-c60e-4fc9-9f1a-95004ea2f396 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.785929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf9c3b-c60e-4fc9-9f1a-95004ea2f396 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.785929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf9c3b-c60e-4fc9-9f1a-95004ea2f396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.860778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-dfbf9c3b-c60e-4fc9-9f1a-95004ea2f396 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:25.862007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dfbf9c3b-c60e-4fc9-9f1a-95004ea2f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:25.955572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 744/1494] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:25.963808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eaf29c-591c-4978-b0cb-000c239cbcd3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.964346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eaf29c-591c-4978-b0cb-000c239cbcd3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.964679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eaf29c-591c-4978-b0cb-000c239cbcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.042214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-b6eaf29c-591c-4978-b0cb-000c239cbcd3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:26.043366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b6eaf29c-591c-4978-b0cb-000c239cbcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:26.136887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 751/1495] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:26.141545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf9c3b-c60e-4fc9-9f1a-95004ea2f396 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.142782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dfbf9c3b-c60e-4fc9-9f1a-95004ea2f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:26.154513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf9c3b-c60e-4fc9-9f1a-95004ea2f396 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.215388 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.249422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.250051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.250327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.250595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.251226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:45:26.251522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.252530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING py.warnings [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:26.252530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: warnings.warn(msg) Sep 13 19:45:26.252530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: Sep 13 19:45:26.253416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.265110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 745/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:45:26.273344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eaf29c-591c-4978-b0cb-000c239cbcd3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.282575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eaf29c-591c-4978-b0cb-000c239cbcd3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.343543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.377359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.377996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.378273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.378540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.379172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.380886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING py.warnings [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:26.380886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: warnings.warn(msg) Sep 13 19:45:26.380886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: Sep 13 19:45:26.382118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.438760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 752/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:45:26 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:26.443160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.444703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:26.477395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb2ce-8939-4b60-8048-a46c7aa640b4 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.569671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.570275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.570552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.570820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.571448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.572449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING py.warnings [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:26.572449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: warnings.warn(msg) Sep 13 19:45:26.572449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: Sep 13 19:45:26.574244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.615125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 746/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:45:26 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:45:26.619417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.621568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:26.682963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2711ee5-6910-4af4-ae72-c165f97f9f9b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.765858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.777130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.777771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.778050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.778320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.778957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.779961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING py.warnings [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:26.779961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: warnings.warn(msg) Sep 13 19:45:26.779961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: Sep 13 19:45:26.780888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.793704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 753/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:45:26.797756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.799302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:26.811363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49777f-808a-4c2d-80a0-36d6d774e2fc admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:26.835734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:26.905397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe455309-6683-46b4-91f8-c4b9bd01d002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:26.905959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe455309-6683-46b4-91f8-c4b9bd01d002 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:26.906238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe455309-6683-46b4-91f8-c4b9bd01d002 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:26.906509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe455309-6683-46b4-91f8-c4b9bd01d002 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:26.907145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe455309-6683-46b4-91f8-c4b9bd01d002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:26.908152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING py.warnings [None req-fe455309-6683-46b4-91f8-c4b9bd01d002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:26.908152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: warnings.warn(msg) Sep 13 19:45:26.908152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: Sep 13 19:45:26.909041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe455309-6683-46b4-91f8-c4b9bd01d002 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:26.921560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 747/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:45:26.925774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:26.927314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:26.960383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d3f58-e619-4e10-9e57-2c1da6f808f8 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:27.063445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f321cd3-66c8-4d65-9257-a664dae4324f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:27.064156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f321cd3-66c8-4d65-9257-a664dae4324f admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:27.064373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f321cd3-66c8-4d65-9257-a664dae4324f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:27.064647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f321cd3-66c8-4d65-9257-a664dae4324f admin admin] PATH_INFO: `/v3/projects/397f001edf024e649feadb0031ecd72b/users/38349ab648dd4b4da391a95cc85a70c2/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:27.086930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f321cd3-66c8-4d65-9257-a664dae4324f admin admin] RBAC: Authorizing `identity:create_grant(project_id=397f001edf024e649feadb0031ecd72b, user_id=38349ab648dd4b4da391a95cc85a70c2, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:27.087984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING py.warnings [None req-8f321cd3-66c8-4d65-9257-a664dae4324f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:27.087984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: warnings.warn(msg) Sep 13 19:45:27.087984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: Sep 13 19:45:27.090509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f321cd3-66c8-4d65-9257-a664dae4324f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:27.116241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 754/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:45:26 2022] PUT /identity/v3/projects/397f001edf024e649feadb0031ecd72b/users/38349ab648dd4b4da391a95cc85a70c2/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:45:27.128399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb565192-5c34-4f4e-b70d-c537c4d46177 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:27.128685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb565192-5c34-4f4e-b70d-c537c4d46177 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:27.128958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb565192-5c34-4f4e-b70d-c537c4d46177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:27.185662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-eb565192-5c34-4f4e-b70d-c537c4d46177 None None] MFA Rules not processed for user `38349ab648dd4b4da391a95cc85a70c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:27.186831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-eb565192-5c34-4f4e-b70d-c537c4d46177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:27.262395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 748/1502] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:45:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:27.275792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c744b5-2c72-402e-98d5-01e9177e34c3 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:27.276595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c744b5-2c72-402e-98d5-01e9177e34c3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:27.276866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c744b5-2c72-402e-98d5-01e9177e34c3 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:27.278259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 755/1503] 10.222.0.23 () {54 vars in 964 bytes} [Tue Sep 13 19:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:27.282830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46c5f-fffb-4d2c-9423-93d6275f7731 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:27.283146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46c5f-fffb-4d2c-9423-93d6275f7731 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:27.283441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b46c5f-fffb-4d2c-9423-93d6275f7731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:27.314689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:27.359413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e2b46c5f-fffb-4d2c-9423-93d6275f7731 None None] MFA Rules not processed for user `284740de651e4065845e945556a05ff4`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:27.360620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e2b46c5f-fffb-4d2c-9423-93d6275f7731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:27.435466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 749/1504] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Sep 13 19:45:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:27.445291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c744b5-2c72-402e-98d5-01e9177e34c3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:27.446775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-86c744b5-2c72-402e-98d5-01e9177e34c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:27.495466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:27.547973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c744b5-2c72-402e-98d5-01e9177e34c3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:27.619427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:27.620071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:27.620362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:27.620630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:27.622236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:27.718427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:27.719765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:27.723279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:27.738026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 756/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:27.784560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183371a-90e7-4418-9453-a9bbca8c73d4 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:27.784849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183371a-90e7-4418-9453-a9bbca8c73d4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:27.785124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183371a-90e7-4418-9453-a9bbca8c73d4 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:27.786417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 750/1506] 10.222.0.23 () {52 vars in 830 bytes} [Tue Sep 13 19:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:27.790110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:27.791312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:27.888649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbc4c3-e8dc-4d5c-89e3-2ff3d4e1e4d8 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:27.979976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:27.980549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:27.980841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:27.981109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] PATH_INFO: `/v3/users/38349ab648dd4b4da391a95cc85a70c2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:27.993537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=38349ab648dd4b4da391a95cc85a70c2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:27.996870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING py.warnings [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:27.996870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: warnings.warn(msg) Sep 13 19:45:27.996870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: Sep 13 19:45:27.998137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:28.008222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:28.065854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38349ab648dd4b4da391a95cc85a70c2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:28.075287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38349ab648dd4b4da391a95cc85a70c2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:28.075744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38349ab648dd4b4da391a95cc85a70c2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:28.098438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38349ab648dd4b4da391a95cc85a70c2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:28.112549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38349ab648dd4b4da391a95cc85a70c2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:28.118282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 757/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:45:27 2022] DELETE /identity/v3/users/38349ab648dd4b4da391a95cc85a70c2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:45:28.123289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266a9a-b32f-4ba7-899a-6dc5ae69982d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:28.123747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266a9a-b32f-4ba7-899a-6dc5ae69982d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:28.123841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266a9a-b32f-4ba7-899a-6dc5ae69982d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:28.179839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8d266a9a-b32f-4ba7-899a-6dc5ae69982d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:28.181032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8d266a9a-b32f-4ba7-899a-6dc5ae69982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:28.255546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:28.268621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 751/1508] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:28.278873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:28.280141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:28.378234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12c41c-bdaa-465b-b298-0960542473ec admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:28.440952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:28.462264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb4a27-7f03-4551-873b-c411e3fde839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:28.462881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4a27-7f03-4551-873b-c411e3fde839 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:28.463154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4a27-7f03-4551-873b-c411e3fde839 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:28.463984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb4a27-7f03-4551-873b-c411e3fde839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:28.465564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3dbb4a27-7f03-4551-873b-c411e3fde839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:28.576578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb4a27-7f03-4551-873b-c411e3fde839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:28.577985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb4a27-7f03-4551-873b-c411e3fde839 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:28.595720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 758/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:28.913231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d266a9a-b32f-4ba7-899a-6dc5ae69982d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:28.914543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8d266a9a-b32f-4ba7-899a-6dc5ae69982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:28.926479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d266a9a-b32f-4ba7-899a-6dc5ae69982d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:29.026024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:29.026725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:29.027008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:29.027281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] PATH_INFO: `/v3/projects/397f001edf024e649feadb0031ecd72b` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:29.037194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=397f001edf024e649feadb0031ecd72b)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:29.038258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING py.warnings [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:29.038258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: warnings.warn(msg) Sep 13 19:45:29.038258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: Sep 13 19:45:29.039473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:29.107110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '397f001edf024e649feadb0031ecd72b', 'request_id': 'req-e296491c-a44a-4a82-ac29-61f9f38b45e2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:29.113093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '397f001edf024e649feadb0031ecd72b', 'request_id': 'req-e296491c-a44a-4a82-ac29-61f9f38b45e2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:29.127939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '397f001edf024e649feadb0031ecd72b', 'request_id': 'req-e296491c-a44a-4a82-ac29-61f9f38b45e2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:29.153202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] The token cache is being invalidate because project 397f001edf024e649feadb0031ecd72b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:45:29.153487 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:29.153946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e296491c-a44a-4a82-ac29-61f9f38b45e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:45:29.155340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 752/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:45:28 2022] DELETE /identity/v3/projects/397f001edf024e649feadb0031ecd72b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:45:38.593467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=62 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:39.151247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:57.491334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04aa8-1448-4a11-b9d1-20c0033e4114 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:57.491958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04aa8-1448-4a11-b9d1-20c0033e4114 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:57.491958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04aa8-1448-4a11-b9d1-20c0033e4114 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:57.493215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:57.498745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0c1c8-a322-4d40-95e7-5d92e0b02bde None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:57.499210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0c1c8-a322-4d40-95e7-5d92e0b02bde None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:57.499319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0c1c8-a322-4d40-95e7-5d92e0b02bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:57.580266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-46e0c1c8-a322-4d40-95e7-5d92e0b02bde None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:57.581480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-46e0c1c8-a322-4d40-95e7-5d92e0b02bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:57.684202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:57.838791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04aa8-1448-4a11-b9d1-20c0033e4114 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:57.840961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-15f04aa8-1448-4a11-b9d1-20c0033e4114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:57.961882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04aa8-1448-4a11-b9d1-20c0033e4114 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:57.967900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef7069-bad4-404a-8ebb-6d00d86ea619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:57.968505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef7069-bad4-404a-8ebb-6d00d86ea619 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:57.968785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef7069-bad4-404a-8ebb-6d00d86ea619 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:57.969065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef7069-bad4-404a-8ebb-6d00d86ea619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:57.977060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef7069-bad4-404a-8ebb-6d00d86ea619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:57.978685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef7069-bad4-404a-8ebb-6d00d86ea619 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:57.987902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 760/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:46:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:46:59.067297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdb06c-43ce-4a39-95ac-0776547e91b7 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:59.067801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdb06c-43ce-4a39-95ac-0776547e91b7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:59.067903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdb06c-43ce-4a39-95ac-0776547e91b7 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:59.069212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:59.076096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975db9f4-9acf-4ecc-b005-0edf908118a9 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:59.076561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975db9f4-9acf-4ecc-b005-0edf908118a9 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:59.076673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975db9f4-9acf-4ecc-b005-0edf908118a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:59.096236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-975db9f4-9acf-4ecc-b005-0edf908118a9 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:59.097424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-975db9f4-9acf-4ecc-b005-0edf908118a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:59.105215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:59.255380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdb06c-43ce-4a39-95ac-0776547e91b7 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:59.270549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdb06c-43ce-4a39-95ac-0776547e91b7 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:59.275977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a66b0-0a3e-4678-acb3-2861c7eb7c94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:59.276571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a66b0-0a3e-4678-acb3-2861c7eb7c94 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:59.276855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a66b0-0a3e-4678-acb3-2861c7eb7c94 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:59.277127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a66b0-0a3e-4678-acb3-2861c7eb7c94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:59.285458 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a66b0-0a3e-4678-acb3-2861c7eb7c94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:59.287099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a66b0-0a3e-4678-acb3-2861c7eb7c94 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:59.296957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 755/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:46:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:46:59.328046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a239f5d-0a1c-4064-9045-b40cf619c74a None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:59.328327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a239f5d-0a1c-4064-9045-b40cf619c74a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:59.328602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a239f5d-0a1c-4064-9045-b40cf619c74a None None] PATH_INFO: `/v3` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:59.330240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 762/1517] 10.222.0.43 () {54 vars in 925 bytes} [Tue Sep 13 19:46:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:59.340611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff35e-366d-498a-9361-6d771bfb6eca None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:59.340900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff35e-366d-498a-9361-6d771bfb6eca None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:59.341180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bff35e-366d-498a-9361-6d771bfb6eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:59.409488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b1bff35e-366d-498a-9361-6d771bfb6eca None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:59.410785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b1bff35e-366d-498a-9361-6d771bfb6eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:59.428666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:59.485152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 756/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Sep 13 19:46:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:47:03.030338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83e38-b1f2-4b03-acd6-22cb2be18e6c None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:03.030866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83e38-b1f2-4b03-acd6-22cb2be18e6c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:03.030866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad83e38-b1f2-4b03-acd6-22cb2be18e6c None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:03.032198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 763/1519] 10.222.0.23 () {54 vars in 961 bytes} [Tue Sep 13 19:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:47:03.037219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79a61-be5c-4198-8b81-8496ec53f68b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:03.037708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79a61-be5c-4198-8b81-8496ec53f68b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:03.037817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f79a61-be5c-4198-8b81-8496ec53f68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:03.057810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-09f79a61-be5c-4198-8b81-8496ec53f68b None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:03.059002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-09f79a61-be5c-4198-8b81-8496ec53f68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:03.067372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 757/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Sep 13 19:47:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:47:03.074857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad83e38-b1f2-4b03-acd6-22cb2be18e6c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:03.089475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad83e38-b1f2-4b03-acd6-22cb2be18e6c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:03.096663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6416d0-a9d7-4930-b9a0-8ed34467b108 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:03.097260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6416d0-a9d7-4930-b9a0-8ed34467b108 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:03.097533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6416d0-a9d7-4930-b9a0-8ed34467b108 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:03.097836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6416d0-a9d7-4930-b9a0-8ed34467b108 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:03.107773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6416d0-a9d7-4930-b9a0-8ed34467b108 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:03.109497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6416d0-a9d7-4930-b9a0-8ed34467b108 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:03.119765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 764/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:47:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:47:10.769428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951210c5-9460-446d-9de1-9e7191963b70 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:10.770083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951210c5-9460-446d-9de1-9e7191963b70 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:10.770083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951210c5-9460-446d-9de1-9e7191963b70 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:10.771350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:47:10.777524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265bc77-c224-4b83-8e6a-8bca6542e53c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:10.778016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265bc77-c224-4b83-8e6a-8bca6542e53c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:10.778120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265bc77-c224-4b83-8e6a-8bca6542e53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:10.797651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4265bc77-c224-4b83-8e6a-8bca6542e53c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:10.798847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4265bc77-c224-4b83-8e6a-8bca6542e53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:10.807014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:47:10.959537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951210c5-9460-446d-9de1-9e7191963b70 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:10.975867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951210c5-9460-446d-9de1-9e7191963b70 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:10.981261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa78668-69e3-4a56-aef2-ff5c398a1de5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:10.982402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78668-69e3-4a56-aef2-ff5c398a1de5 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:10.982680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78668-69e3-4a56-aef2-ff5c398a1de5 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:10.982946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa78668-69e3-4a56-aef2-ff5c398a1de5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:10.991273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa78668-69e3-4a56-aef2-ff5c398a1de5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:10.992848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa78668-69e3-4a56-aef2-ff5c398a1de5 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:11.003681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 759/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:47:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:47:12.074226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8a2c-8ba7-4587-9793-e9aa14161348 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:12.074713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8a2c-8ba7-4587-9793-e9aa14161348 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:12.074823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8a2c-8ba7-4587-9793-e9aa14161348 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:12.076204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:47:12.082385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de0494-6eda-4f33-b505-b0f85adfe7ff None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:12.082853 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de0494-6eda-4f33-b505-b0f85adfe7ff None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:12.082956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de0494-6eda-4f33-b505-b0f85adfe7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:12.102860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-72de0494-6eda-4f33-b505-b0f85adfe7ff None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:12.104064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-72de0494-6eda-4f33-b505-b0f85adfe7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:12.112518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:47:12.263510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de8a2c-8ba7-4587-9793-e9aa14161348 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:12.277911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de8a2c-8ba7-4587-9793-e9aa14161348 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:12.283962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec829707-7041-4469-af6b-afbdb185b255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:12.284558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829707-7041-4469-af6b-afbdb185b255 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:12.284835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829707-7041-4469-af6b-afbdb185b255 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:12.285105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829707-7041-4469-af6b-afbdb185b255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:12.293131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec829707-7041-4469-af6b-afbdb185b255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:12.294734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec829707-7041-4469-af6b-afbdb185b255 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:12.304165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 767/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:47:16.326137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49b3e0-ec44-4183-b0b3-f0391bc99dd2 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:16.326711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49b3e0-ec44-4183-b0b3-f0391bc99dd2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:16.326711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49b3e0-ec44-4183-b0b3-f0391bc99dd2 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:16.327982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:47:16.333072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd838bc5-11b5-4498-ab7a-df192ec18e52 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:16.333481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd838bc5-11b5-4498-ab7a-df192ec18e52 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:16.333628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd838bc5-11b5-4498-ab7a-df192ec18e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:16.354322 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bd838bc5-11b5-4498-ab7a-df192ec18e52 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:16.355510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bd838bc5-11b5-4498-ab7a-df192ec18e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:16.363821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:47:16.652553 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49b3e0-ec44-4183-b0b3-f0391bc99dd2 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:16.654316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9b49b3e0-ec44-4183-b0b3-f0391bc99dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:16.734173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49b3e0-ec44-4183-b0b3-f0391bc99dd2 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:16.737703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c9c83-cd9a-45cb-9f19-fd157a5b58c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:16.738307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9c83-cd9a-45cb-9f19-fd157a5b58c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:16.738583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9c83-cd9a-45cb-9f19-fd157a5b58c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:16.738854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c9c83-cd9a-45cb-9f19-fd157a5b58c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:16.748158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c9c83-cd9a-45cb-9f19-fd157a5b58c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:16.749573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c9c83-cd9a-45cb-9f19-fd157a5b58c2 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:16.759045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 762/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:47:18.906458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7850c-e618-4c02-a4cf-c4928184beab None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:18.907013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7850c-e618-4c02-a4cf-c4928184beab None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:18.907013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7850c-e618-4c02-a4cf-c4928184beab None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:18.908347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:47:18.913824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f980466-e807-4153-a073-33d6c1fa8c09 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:18.914269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f980466-e807-4153-a073-33d6c1fa8c09 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:18.914405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f980466-e807-4153-a073-33d6c1fa8c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:18.933571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-9f980466-e807-4153-a073-33d6c1fa8c09 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:18.934810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9f980466-e807-4153-a073-33d6c1fa8c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:18.942859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:47:19.230460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7850c-e618-4c02-a4cf-c4928184beab None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:19.235887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=75 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:19.242389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7850c-e618-4c02-a4cf-c4928184beab None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:19.245937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3568d-ea67-4d0d-abcf-bc515c7f6407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:19.246528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3568d-ea67-4d0d-abcf-bc515c7f6407 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:19.247304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3568d-ea67-4d0d-abcf-bc515c7f6407 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:19.247570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3568d-ea67-4d0d-abcf-bc515c7f6407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:19.256280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3568d-ea67-4d0d-abcf-bc515c7f6407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:19.257720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c3568d-ea67-4d0d-abcf-bc515c7f6407 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:19.267354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 770/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:47:23.085341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96433bec-006f-4ffc-a0da-7c0bf059b876 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:23.085966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96433bec-006f-4ffc-a0da-7c0bf059b876 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:23.085966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96433bec-006f-4ffc-a0da-7c0bf059b876 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:23.087286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:47:23.092631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7a808-41e9-4be1-a5c3-94ca2b7c2267 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:23.093074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7a808-41e9-4be1-a5c3-94ca2b7c2267 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:23.093177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7a808-41e9-4be1-a5c3-94ca2b7c2267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:23.112133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9ef7a808-41e9-4be1-a5c3-94ca2b7c2267 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:23.113317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9ef7a808-41e9-4be1-a5c3-94ca2b7c2267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:23.121086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:47:23.421567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96433bec-006f-4ffc-a0da-7c0bf059b876 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:23.423263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-96433bec-006f-4ffc-a0da-7c0bf059b876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:23.487498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:23.501889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96433bec-006f-4ffc-a0da-7c0bf059b876 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:23.505436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b005baf-35e5-48ff-b7d0-28854ab064a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:23.506072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b005baf-35e5-48ff-b7d0-28854ab064a5 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:23.506353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b005baf-35e5-48ff-b7d0-28854ab064a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:23.506625 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b005baf-35e5-48ff-b7d0-28854ab064a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:23.515341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b005baf-35e5-48ff-b7d0-28854ab064a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:23.516844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b005baf-35e5-48ff-b7d0-28854ab064a5 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:23.526453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 765/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:47:25.663179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe31d54-f604-4a57-872f-625de0a63bb2 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:25.663722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe31d54-f604-4a57-872f-625de0a63bb2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:25.663722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe31d54-f604-4a57-872f-625de0a63bb2 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:25.665024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:47:25.670721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9bd98-993a-41af-92db-fac53dbb2ea9 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:25.671155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9bd98-993a-41af-92db-fac53dbb2ea9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:25.671260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9bd98-993a-41af-92db-fac53dbb2ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:25.691266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-07a9bd98-993a-41af-92db-fac53dbb2ea9 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:47:25.692448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-07a9bd98-993a-41af-92db-fac53dbb2ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:25.700103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:47:25.998267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe31d54-f604-4a57-872f-625de0a63bb2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:47:25.999950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-dfe31d54-f604-4a57-872f-625de0a63bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:47:26.012915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe31d54-f604-4a57-872f-625de0a63bb2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:47:26.016631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3503-a5f9-4dae-b0ad-b19805429db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:47:26.017235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3503-a5f9-4dae-b0ad-b19805429db9 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:47:26.017509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3503-a5f9-4dae-b0ad-b19805429db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:47:26.017814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3503-a5f9-4dae-b0ad-b19805429db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:47:26.026987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3503-a5f9-4dae-b0ad-b19805429db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:47:26.028910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3503-a5f9-4dae-b0ad-b19805429db9 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:47:26.038186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 773/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:47:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:47:35.686516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=11 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:47:36.034013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=14 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:50.388705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62e084-0a0b-447b-9dbf-2ebad5067f54 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.389421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62e084-0a0b-447b-9dbf-2ebad5067f54 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.389421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62e084-0a0b-447b-9dbf-2ebad5067f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.412666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7e62e084-0a0b-447b-9dbf-2ebad5067f54 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:50.413911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7e62e084-0a0b-447b-9dbf-2ebad5067f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:50.428071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 767/1540] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:50.437730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa591b6-cf65-45a2-b306-47ab19149482 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.438350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa591b6-cf65-45a2-b306-47ab19149482 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.438350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa591b6-cf65-45a2-b306-47ab19149482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.471580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9aa591b6-cf65-45a2-b306-47ab19149482 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:50.472833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9aa591b6-cf65-45a2-b306-47ab19149482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:50.484940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 774/1541] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:50.489545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62e084-0a0b-447b-9dbf-2ebad5067f54 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:50.507502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62e084-0a0b-447b-9dbf-2ebad5067f54 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:50.516475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:50.517111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.517685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.517685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.518477 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:50.518667 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:50.520288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:50.543988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 768/1542] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:50.552659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa591b6-cf65-45a2-b306-47ab19149482 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:50.562414 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa591b6-cf65-45a2-b306-47ab19149482 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:50.569966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7583dca-ae55-4b38-a0d7-91b7a2e6d585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:50.570571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583dca-ae55-4b38-a0d7-91b7a2e6d585 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.570852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583dca-ae55-4b38-a0d7-91b7a2e6d585 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.571671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7583dca-ae55-4b38-a0d7-91b7a2e6d585 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.572315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7583dca-ae55-4b38-a0d7-91b7a2e6d585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:50.574292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7583dca-ae55-4b38-a0d7-91b7a2e6d585 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:50.630232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 775/1543] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/projects => generated 393 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:50.635269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:50.647844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ef660-44d5-4b6d-9990-1993a78bce94 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:50.730355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c88a6-c3b6-4460-a52c-934b2c9c2f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:50.730976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c88a6-c3b6-4460-a52c-934b2c9c2f1d admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.731255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c88a6-c3b6-4460-a52c-934b2c9c2f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.731657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c88a6-c3b6-4460-a52c-934b2c9c2f1d admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.732306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c88a6-c3b6-4460-a52c-934b2c9c2f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:50.734452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c88a6-c3b6-4460-a52c-934b2c9c2f1d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:50.771186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 769/1544] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:50.775087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7583dca-ae55-4b38-a0d7-91b7a2e6d585 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:50.783490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce318a9-fafc-4843-a9e2-db943f3c6f62 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.783868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce318a9-fafc-4843-a9e2-db943f3c6f62 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.784258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce318a9-fafc-4843-a9e2-db943f3c6f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.787242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7583dca-ae55-4b38-a0d7-91b7a2e6d585 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:50.797554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb2f9c-e0fc-4555-9a7a-783a3b6cbb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:50.798213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2f9c-e0fc-4555-9a7a-783a3b6cbb47 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.798500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2f9c-e0fc-4555-9a7a-783a3b6cbb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.798778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb2f9c-e0fc-4555-9a7a-783a3b6cbb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.799432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb2f9c-e0fc-4555-9a7a-783a3b6cbb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:50.801382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb2f9c-e0fc-4555-9a7a-783a3b6cbb47 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:50.814013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8ce318a9-fafc-4843-a9e2-db943f3c6f62 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:50.815388 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8ce318a9-fafc-4843-a9e2-db943f3c6f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:50.822502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 776/1545] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:50.833353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb2f9c-e0fc-4555-9a7a-783a3b6cbb47 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:50.837128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 770/1546] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:50.846941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb2f9c-e0fc-4555-9a7a-783a3b6cbb47 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:50.847723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5df5d2-db1c-4cbe-9c51-57ae71e69983 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.848064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5df5d2-db1c-4cbe-9c51-57ae71e69983 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.848417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5df5d2-db1c-4cbe-9c51-57ae71e69983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.858401 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e2c3f-a4a7-4ab1-ae29-09bde1a19ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:50.859044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e2c3f-a4a7-4ab1-ae29-09bde1a19ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.859328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e2c3f-a4a7-4ab1-ae29-09bde1a19ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.859604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e2c3f-a4a7-4ab1-ae29-09bde1a19ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.860263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e2c3f-a4a7-4ab1-ae29-09bde1a19ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:50.861909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e2c3f-a4a7-4ab1-ae29-09bde1a19ee3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:50.882136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-0d5df5d2-db1c-4cbe-9c51-57ae71e69983 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:50.883519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0d5df5d2-db1c-4cbe-9c51-57ae71e69983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:50.883899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 777/1547] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:50.885292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d86aa-c4d9-44b5-a8d3-2c79bb4f8240 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.885584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d86aa-c4d9-44b5-a8d3-2c79bb4f8240 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.885892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d86aa-c4d9-44b5-a8d3-2c79bb4f8240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.911445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 771/1548] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:50.912253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5df5d2-db1c-4cbe-9c51-57ae71e69983 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:50.914159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-115d86aa-c4d9-44b5-a8d3-2c79bb4f8240 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:50.915485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-115d86aa-c4d9-44b5-a8d3-2c79bb4f8240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:50.926708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5df5d2-db1c-4cbe-9c51-57ae71e69983 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:50.932973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 778/1549] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:50.934286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3e99-45c5-423d-8643-cf691872040c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.934572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3e99-45c5-423d-8643-cf691872040c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.934846 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3e99-45c5-423d-8643-cf691872040c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.944420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:50.945114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.945402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.945709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] PATH_INFO: `/v3/projects/6991f30f2c094499b73579677b12abae/users/a0e295fda2af4157b196c6130aa2d6d2/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.965857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-054a3e99-45c5-423d-8643-cf691872040c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:50.967082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-054a3e99-45c5-423d-8643-cf691872040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:50.967871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6991f30f2c094499b73579677b12abae, user_id=a0e295fda2af4157b196c6130aa2d6d2, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:50.969494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING py.warnings [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:48:50.969494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: warnings.warn(msg) Sep 13 19:48:50.969494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: Sep 13 19:48:50.972082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:50.975443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 779/1550] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:50.981630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55248b-1112-4c27-910e-ce8c082d997c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:50.981630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55248b-1112-4c27-910e-ce8c082d997c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:50.981630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55248b-1112-4c27-910e-ce8c082d997c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:50.998667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ca55248b-1112-4c27-910e-ce8c082d997c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:50.999047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 772/1551] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:50 2022] PUT /identity/v3/projects/6991f30f2c094499b73579677b12abae/users/a0e295fda2af4157b196c6130aa2d6d2/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:50.999833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.000959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ca55248b-1112-4c27-910e-ce8c082d997c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.073275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a55da-3a3a-4a57-92d4-2ce4883ceff9 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.078688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41891c5b-00d3-4b8c-8746-b4b4bfa99d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.079315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891c5b-00d3-4b8c-8746-b4b4bfa99d7d admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.079582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891c5b-00d3-4b8c-8746-b4b4bfa99d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.079858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891c5b-00d3-4b8c-8746-b4b4bfa99d7d admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.080643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41891c5b-00d3-4b8c-8746-b4b4bfa99d7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:51.080957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41891c5b-00d3-4b8c-8746-b4b4bfa99d7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.083095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41891c5b-00d3-4b8c-8746-b4b4bfa99d7d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.101647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 773/1552] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:51.101647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa5067-3794-4ed9-b1b2-cc3106b014e2 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.101647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa5067-3794-4ed9-b1b2-cc3106b014e2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.101647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa5067-3794-4ed9-b1b2-cc3106b014e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.104960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 780/1553] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.106815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d1acf-3519-4950-993d-171da3323057 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.107100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d1acf-3519-4950-993d-171da3323057 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.107409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d1acf-3519-4950-993d-171da3323057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.128352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f23d1acf-3519-4950-993d-171da3323057 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.128563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-39aa5067-3794-4ed9-b1b2-cc3106b014e2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.129555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f23d1acf-3519-4950-993d-171da3323057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.129729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-39aa5067-3794-4ed9-b1b2-cc3106b014e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.137818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 774/1554] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.145482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 781/1555] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.145718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db09e0-e151-4269-b81a-4ef6803d37aa None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.145718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db09e0-e151-4269-b81a-4ef6803d37aa None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.145919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ff5a1-c7bc-48a6-b85d-96d67f728dbe None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.146012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db09e0-e151-4269-b81a-4ef6803d37aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.146118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ff5a1-c7bc-48a6-b85d-96d67f728dbe None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.146118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ff5a1-c7bc-48a6-b85d-96d67f728dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.162131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5c9ff5a1-c7bc-48a6-b85d-96d67f728dbe None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.162975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c5db09e0-e151-4269-b81a-4ef6803d37aa None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.163269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5c9ff5a1-c7bc-48a6-b85d-96d67f728dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.164187 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c5db09e0-e151-4269-b81a-4ef6803d37aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.171453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 782/1556] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.176252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 775/1557] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.176896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b048e-7e0d-4887-a15d-c218ea7cdeef None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.176896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b048e-7e0d-4887-a15d-c218ea7cdeef None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.176896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b048e-7e0d-4887-a15d-c218ea7cdeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.177140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db09e0-e151-4269-b81a-4ef6803d37aa None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.187428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db09e0-e151-4269-b81a-4ef6803d37aa None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.192227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42992ce-ed95-47ab-98a1-89599215d272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.192824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42992ce-ed95-47ab-98a1-89599215d272 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.193102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42992ce-ed95-47ab-98a1-89599215d272 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.193373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42992ce-ed95-47ab-98a1-89599215d272 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.193516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ad8b048e-7e0d-4887-a15d-c218ea7cdeef None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.194031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42992ce-ed95-47ab-98a1-89599215d272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.194753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ad8b048e-7e0d-4887-a15d-c218ea7cdeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.195129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING py.warnings [None req-e42992ce-ed95-47ab-98a1-89599215d272 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:48:51.195129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: warnings.warn(msg) Sep 13 19:48:51.195129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: Sep 13 19:48:51.196357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42992ce-ed95-47ab-98a1-89599215d272 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.209791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 783/1558] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.209791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b048e-7e0d-4887-a15d-c218ea7cdeef None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.210789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:51.224530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b048e-7e0d-4887-a15d-c218ea7cdeef None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.230539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 776/1559] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:51.230539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59e54c-5797-44fd-98ca-a0aa30d846fb None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.230539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59e54c-5797-44fd-98ca-a0aa30d846fb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.230539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e59e54c-5797-44fd-98ca-a0aa30d846fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.232641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d9746-22cb-4357-bdf2-449bf2c15570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.233251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9746-22cb-4357-bdf2-449bf2c15570 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.233530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9746-22cb-4357-bdf2-449bf2c15570 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.233833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d9746-22cb-4357-bdf2-449bf2c15570 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.234482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d9746-22cb-4357-bdf2-449bf2c15570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:51.234787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d9746-22cb-4357-bdf2-449bf2c15570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.235833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING py.warnings [None req-986d9746-22cb-4357-bdf2-449bf2c15570 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:48:51.235833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: warnings.warn(msg) Sep 13 19:48:51.235833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: Sep 13 19:48:51.236728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d9746-22cb-4357-bdf2-449bf2c15570 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.248745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3e59e54c-5797-44fd-98ca-a0aa30d846fb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.256177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 784/1560] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:51.256343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3e59e54c-5797-44fd-98ca-a0aa30d846fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.256443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa66b5-d910-4ef0-9e90-ab4a7bb0e805 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.256443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa66b5-d910-4ef0-9e90-ab4a7bb0e805 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.256443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa66b5-d910-4ef0-9e90-ab4a7bb0e805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.270910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4bfa66b5-d910-4ef0-9e90-ab4a7bb0e805 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.272120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4bfa66b5-d910-4ef0-9e90-ab4a7bb0e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.311565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 777/1561] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.319065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477711d-8c1e-43c8-a5b3-6ef845e16ee3 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.319419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477711d-8c1e-43c8-a5b3-6ef845e16ee3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.319726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477711d-8c1e-43c8-a5b3-6ef845e16ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.320885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:51.334013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 785/1562] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.334914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa66b5-d910-4ef0-9e90-ab4a7bb0e805 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.339213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1477711d-8c1e-43c8-a5b3-6ef845e16ee3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.340414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1477711d-8c1e-43c8-a5b3-6ef845e16ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.344287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa66b5-d910-4ef0-9e90-ab4a7bb0e805 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.348362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 778/1563] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.348772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3db8cd2-5b89-499f-9b75-b5fa3d9a2553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.349350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477711d-8c1e-43c8-a5b3-6ef845e16ee3 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.349467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db8cd2-5b89-499f-9b75-b5fa3d9a2553 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.349720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db8cd2-5b89-499f-9b75-b5fa3d9a2553 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.350000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3db8cd2-5b89-499f-9b75-b5fa3d9a2553 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.350634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db8cd2-5b89-499f-9b75-b5fa3d9a2553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:51.351055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db8cd2-5b89-499f-9b75-b5fa3d9a2553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.353392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3db8cd2-5b89-499f-9b75-b5fa3d9a2553 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.365008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 786/1564] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:51.366231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7dbf5-5885-4eb0-8ca9-59a5f9d13c3b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.366513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7dbf5-5885-4eb0-8ca9-59a5f9d13c3b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.366785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7dbf5-5885-4eb0-8ca9-59a5f9d13c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.386960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-55e7dbf5-5885-4eb0-8ca9-59a5f9d13c3b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.388155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-55e7dbf5-5885-4eb0-8ca9-59a5f9d13c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.396168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 787/1565] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.396886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7dbf5-5885-4eb0-8ca9-59a5f9d13c3b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.406099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7dbf5-5885-4eb0-8ca9-59a5f9d13c3b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.406793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477711d-8c1e-43c8-a5b3-6ef845e16ee3 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.410612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a51b8-208f-4199-9c5d-8f795eca6059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.411090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc97597-9369-4cca-9c50-4b981edba1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.411222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a51b8-208f-4199-9c5d-8f795eca6059 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.411512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a51b8-208f-4199-9c5d-8f795eca6059 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.411678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97597-9369-4cca-9c50-4b981edba1df admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.411776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a51b8-208f-4199-9c5d-8f795eca6059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.411954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97597-9369-4cca-9c50-4b981edba1df admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.412274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc97597-9369-4cca-9c50-4b981edba1df admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.412843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97597-9369-4cca-9c50-4b981edba1df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.413912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING py.warnings [None req-4bc97597-9369-4cca-9c50-4b981edba1df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:48:51.413912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: warnings.warn(msg) Sep 13 19:48:51.413912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: Sep 13 19:48:51.415633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc97597-9369-4cca-9c50-4b981edba1df admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.421804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a51b8-208f-4199-9c5d-8f795eca6059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.423303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a51b8-208f-4199-9c5d-8f795eca6059 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.434533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 779/1566] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:51.435231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a51b8-208f-4199-9c5d-8f795eca6059 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.440702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 788/1567] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:51.440702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea338ba-665d-460d-bba2-d15915c8b7e1 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.440702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea338ba-665d-460d-bba2-d15915c8b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.440702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea338ba-665d-460d-bba2-d15915c8b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.445145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a51b8-208f-4199-9c5d-8f795eca6059 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.450771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194f02b-70ea-4d30-840b-412bc34c9aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.451831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194f02b-70ea-4d30-840b-412bc34c9aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.451968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194f02b-70ea-4d30-840b-412bc34c9aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.452298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194f02b-70ea-4d30-840b-412bc34c9aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.453032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194f02b-70ea-4d30-840b-412bc34c9aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.455383 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194f02b-70ea-4d30-840b-412bc34c9aa2 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.460667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4ea338ba-665d-460d-bba2-d15915c8b7e1 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:51.462507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4ea338ba-665d-460d-bba2-d15915c8b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:51.476768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 789/1568] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:51.478161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea338ba-665d-460d-bba2-d15915c8b7e1 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.488996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea338ba-665d-460d-bba2-d15915c8b7e1 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.491218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 780/1569] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:51.494826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194f02b-70ea-4d30-840b-412bc34c9aa2 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.495232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.495911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.496187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.496449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.497155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:51.497419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.499105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.511420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 790/1570] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:51.518595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.533092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c6d70-5c1a-4285-a953-a05da9b15b7a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.575387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194f02b-70ea-4d30-840b-412bc34c9aa2 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.580297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.580994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.581393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.581778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.582510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:51.582886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.584663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.594603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:51.605448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 781/1571] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:51.606422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.611119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c78f-2a4e-4b37-ada6-7fb0b1a53c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.611880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c78f-2a4e-4b37-ada6-7fb0b1a53c7f admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.612232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c78f-2a4e-4b37-ada6-7fb0b1a53c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.612639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2c78f-2a4e-4b37-ada6-7fb0b1a53c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.613203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2c78f-2a4e-4b37-ada6-7fb0b1a53c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.618845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f3a11-ec28-42be-88f0-04824ccc7a9f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.619190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2c78f-2a4e-4b37-ada6-7fb0b1a53c7f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.623905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.624262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.624441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.624950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.625489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:51.625836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.627449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.644311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 782/1572] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:51.644548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.655829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023db919-cb8f-4f6a-94f4-21a9575df2ed admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.661692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 791/1573] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/projects => generated 355 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:51.661871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6d0bd-c39d-40aa-b9d5-a64f96e711b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.661989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c78f-2a4e-4b37-ada6-7fb0b1a53c7f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.662088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6d0bd-c39d-40aa-b9d5-a64f96e711b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.662088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6d0bd-c39d-40aa-b9d5-a64f96e711b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.662088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6d0bd-c39d-40aa-b9d5-a64f96e711b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.662582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6d0bd-c39d-40aa-b9d5-a64f96e711b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.664154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6d0bd-c39d-40aa-b9d5-a64f96e711b7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.674515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 783/1574] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:51.678533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6d0bd-c39d-40aa-b9d5-a64f96e711b7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.689400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6d0bd-c39d-40aa-b9d5-a64f96e711b7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.737963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2c78f-2a4e-4b37-ada6-7fb0b1a53c7f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.743553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.744168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.744454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.744723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.745369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:48:51.745703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.747310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.750743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2ab0-19ea-4bda-9c04-6481b034d058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.751569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2ab0-19ea-4bda-9c04-6481b034d058 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.751998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2ab0-19ea-4bda-9c04-6481b034d058 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.752432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2ab0-19ea-4bda-9c04-6481b034d058 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.753208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a2ab0-19ea-4bda-9c04-6481b034d058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.755373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a2ab0-19ea-4bda-9c04-6481b034d058 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.761077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 792/1575] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:51.763088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.776757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51325bda-d997-47e1-81cb-2e75a49cf32d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.780924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 784/1576] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:51.781836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2ab0-19ea-4bda-9c04-6481b034d058 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.782283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b3b9-d86b-49de-83cc-73b4fc9b4620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.782993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b3b9-d86b-49de-83cc-73b4fc9b4620 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.783388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b3b9-d86b-49de-83cc-73b4fc9b4620 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.784068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b3b9-d86b-49de-83cc-73b4fc9b4620 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.784706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52b3b9-d86b-49de-83cc-73b4fc9b4620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.786663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52b3b9-d86b-49de-83cc-73b4fc9b4620 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.794868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2ab0-19ea-4bda-9c04-6481b034d058 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.800345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a52c7b-b22b-4d52-8dd8-446e205bbc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.801646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a52c7b-b22b-4d52-8dd8-446e205bbc63 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.802135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a52c7b-b22b-4d52-8dd8-446e205bbc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.802569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a52c7b-b22b-4d52-8dd8-446e205bbc63 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.803694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a52c7b-b22b-4d52-8dd8-446e205bbc63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.804884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:51.808103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a52c7b-b22b-4d52-8dd8-446e205bbc63 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.822244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 793/1577] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:51.822244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b3b9-d86b-49de-83cc-73b4fc9b4620 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.866861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 785/1578] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:51.867440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a52c7b-b22b-4d52-8dd8-446e205bbc63 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.900597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b3b9-d86b-49de-83cc-73b4fc9b4620 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.939764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a52c7b-b22b-4d52-8dd8-446e205bbc63 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.946669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa281b4-45b6-4934-a08a-790105ad9404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.947287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa281b4-45b6-4934-a08a-790105ad9404 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.947614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa281b4-45b6-4934-a08a-790105ad9404 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.947906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa281b4-45b6-4934-a08a-790105ad9404 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.948606 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa281b4-45b6-4934-a08a-790105ad9404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.950987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa281b4-45b6-4934-a08a-790105ad9404 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.972644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13310bf9-815d-4403-aee9-bbbc792fb148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.973267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310bf9-815d-4403-aee9-bbbc792fb148 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.973573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310bf9-815d-4403-aee9-bbbc792fb148 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.974048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310bf9-815d-4403-aee9-bbbc792fb148 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.974680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13310bf9-815d-4403-aee9-bbbc792fb148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.976659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13310bf9-815d-4403-aee9-bbbc792fb148 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:51.978290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 786/1579] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/users => generated 278 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:51.978976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa281b4-45b6-4934-a08a-790105ad9404 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:51.989145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa281b4-45b6-4934-a08a-790105ad9404 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:51.994559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a74ee-51d3-49e1-8bb2-96d7bb1555d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:51.995166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a74ee-51d3-49e1-8bb2-96d7bb1555d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:51.995450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a74ee-51d3-49e1-8bb2-96d7bb1555d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:51.995727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a74ee-51d3-49e1-8bb2-96d7bb1555d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:51.996371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a74ee-51d3-49e1-8bb2-96d7bb1555d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:51.998042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a74ee-51d3-49e1-8bb2-96d7bb1555d3 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.010961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 794/1580] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:51 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:52.010961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13310bf9-815d-4403-aee9-bbbc792fb148 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.011265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 787/1581] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.013262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a74ee-51d3-49e1-8bb2-96d7bb1555d3 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.024149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a74ee-51d3-49e1-8bb2-96d7bb1555d3 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.048410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:52.093972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ba09-5c2c-47b6-a5ba-49f3b52bf33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.094589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ba09-5c2c-47b6-a5ba-49f3b52bf33a admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.094873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ba09-5c2c-47b6-a5ba-49f3b52bf33a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.095243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a881ba09-5c2c-47b6-a5ba-49f3b52bf33a admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.096096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881ba09-5c2c-47b6-a5ba-49f3b52bf33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.097517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a881ba09-5c2c-47b6-a5ba-49f3b52bf33a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.102180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13310bf9-815d-4403-aee9-bbbc792fb148 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.110179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 788/1582] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.111462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ba09-5c2c-47b6-a5ba-49f3b52bf33a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.118274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec85586-e906-412a-b987-4c7ffd5bb83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.119785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85586-e906-412a-b987-4c7ffd5bb83e admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.120082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85586-e906-412a-b987-4c7ffd5bb83e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.120359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85586-e906-412a-b987-4c7ffd5bb83e admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.121047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec85586-e906-412a-b987-4c7ffd5bb83e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.123093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec85586-e906-412a-b987-4c7ffd5bb83e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.123522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a881ba09-5c2c-47b6-a5ba-49f3b52bf33a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.128225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251804f-34ef-4d3a-9294-fb8705c4dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.128930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251804f-34ef-4d3a-9294-fb8705c4dcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.129194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251804f-34ef-4d3a-9294-fb8705c4dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.129474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251804f-34ef-4d3a-9294-fb8705c4dcf9 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.130259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251804f-34ef-4d3a-9294-fb8705c4dcf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.132331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251804f-34ef-4d3a-9294-fb8705c4dcf9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.158072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 789/1583] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:52 2022] POST /identity/v3/users => generated 295 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:52.162203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 795/1584] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:52 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:52.162362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251804f-34ef-4d3a-9294-fb8705c4dcf9 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.162450 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec85586-e906-412a-b987-4c7ffd5bb83e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.173902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251804f-34ef-4d3a-9294-fb8705c4dcf9 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.178767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:52.255509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec85586-e906-412a-b987-4c7ffd5bb83e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.264987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f2fec-89c8-47d8-bb76-e9519f434113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.266064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2fec-89c8-47d8-bb76-e9519f434113 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.266675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2fec-89c8-47d8-bb76-e9519f434113 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.266956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f2fec-89c8-47d8-bb76-e9519f434113 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.267668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f2fec-89c8-47d8-bb76-e9519f434113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.270778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f2fec-89c8-47d8-bb76-e9519f434113 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.281241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec297ad-5594-4166-80a6-d07630532df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.283141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec297ad-5594-4166-80a6-d07630532df5 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.283652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec297ad-5594-4166-80a6-d07630532df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.283781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 796/1585] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.284343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec297ad-5594-4166-80a6-d07630532df5 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.284531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f2fec-89c8-47d8-bb76-e9519f434113 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.285591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec297ad-5594-4166-80a6-d07630532df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.288345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec297ad-5594-4166-80a6-d07630532df5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.297381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f2fec-89c8-47d8-bb76-e9519f434113 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.302140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f371dba-6c79-49d2-96cd-f44359f280d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.302872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f371dba-6c79-49d2-96cd-f44359f280d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.303254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f371dba-6c79-49d2-96cd-f44359f280d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.303627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f371dba-6c79-49d2-96cd-f44359f280d9 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.304488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f371dba-6c79-49d2-96cd-f44359f280d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.307856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f371dba-6c79-49d2-96cd-f44359f280d9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.330267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 790/1586] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:52 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:52.330267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec297ad-5594-4166-80a6-d07630532df5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.343673 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec297ad-5594-4166-80a6-d07630532df5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.347296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 797/1587] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:52 2022] POST /identity/v3/users => generated 287 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:52.348271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f371dba-6c79-49d2-96cd-f44359f280d9 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.349803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6cf2-998d-41cf-b02c-642916f7c9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.350692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6cf2-998d-41cf-b02c-642916f7c9a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.351255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6cf2-998d-41cf-b02c-642916f7c9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.351555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b6cf2-998d-41cf-b02c-642916f7c9a5 admin admin] PATH_INFO: `/v3/projects/f2c073eb9c7a494ab879f3af8cc96717/users/0cf74b3dd4c24c1aa3c0ac659b37c0e1/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.360585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f371dba-6c79-49d2-96cd-f44359f280d9 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.374844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7cfed-6124-4b3d-835b-d0779eae7669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.375719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7cfed-6124-4b3d-835b-d0779eae7669 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.375719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7cfed-6124-4b3d-835b-d0779eae7669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.375997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7cfed-6124-4b3d-835b-d0779eae7669 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.376633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7cfed-6124-4b3d-835b-d0779eae7669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.378922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7cfed-6124-4b3d-835b-d0779eae7669 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.384509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b6cf2-998d-41cf-b02c-642916f7c9a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c073eb9c7a494ab879f3af8cc96717, user_id=0cf74b3dd4c24c1aa3c0ac659b37c0e1, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.387068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b6cf2-998d-41cf-b02c-642916f7c9a5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.403483 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 791/1588] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:52 2022] PUT /identity/v3/projects/f2c073eb9c7a494ab879f3af8cc96717/users/0cf74b3dd4c24c1aa3c0ac659b37c0e1/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:52.404449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6cf2-998d-41cf-b02c-642916f7c9a5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.404965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 798/1589] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.406445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7cfed-6124-4b3d-835b-d0779eae7669 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.419723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7cfed-6124-4b3d-835b-d0779eae7669 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.477058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b6cf2-998d-41cf-b02c-642916f7c9a5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.487484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4732b-1315-4ecd-89e0-53384851d935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.488103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4732b-1315-4ecd-89e0-53384851d935 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.488375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4732b-1315-4ecd-89e0-53384851d935 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.488652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4732b-1315-4ecd-89e0-53384851d935 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.489292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4732b-1315-4ecd-89e0-53384851d935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.490966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4732b-1315-4ecd-89e0-53384851d935 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.502008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cc6e3-ab8e-4eb4-b4cc-b9954bfb67c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.502575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cc6e3-ab8e-4eb4-b4cc-b9954bfb67c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.502662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cc6e3-ab8e-4eb4-b4cc-b9954bfb67c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.502898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cc6e3-ab8e-4eb4-b4cc-b9954bfb67c7 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.503516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cc6e3-ab8e-4eb4-b4cc-b9954bfb67c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.505440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cc6e3-ab8e-4eb4-b4cc-b9954bfb67c7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.512546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 792/1590] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.513655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4732b-1315-4ecd-89e0-53384851d935 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.522636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:52.530117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4732b-1315-4ecd-89e0-53384851d935 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.538468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 799/1591] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:52 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:52.538468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cc6e3-ab8e-4eb4-b4cc-b9954bfb67c7 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.539870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485a585-d468-4438-849a-0d57571d0df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.542332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485a585-d468-4438-849a-0d57571d0df7 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.542772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485a585-d468-4438-849a-0d57571d0df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.543229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485a585-d468-4438-849a-0d57571d0df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.544087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485a585-d468-4438-849a-0d57571d0df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.545947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5485a585-d468-4438-849a-0d57571d0df7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.554128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cc6e3-ab8e-4eb4-b4cc-b9954bfb67c7 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.560558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb14df8-6a09-40b0-bb57-3510cec59379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.561345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 793/1592] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.562470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485a585-d468-4438-849a-0d57571d0df7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.564992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb14df8-6a09-40b0-bb57-3510cec59379 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.565426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb14df8-6a09-40b0-bb57-3510cec59379 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.565819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb14df8-6a09-40b0-bb57-3510cec59379 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.566623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb14df8-6a09-40b0-bb57-3510cec59379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.568443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb14df8-6a09-40b0-bb57-3510cec59379 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.574201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485a585-d468-4438-849a-0d57571d0df7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.580825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5207350-ffc0-4421-bf71-5fdaf07cc22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.581862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5207350-ffc0-4421-bf71-5fdaf07cc22b admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.582076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 800/1593] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.582191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5207350-ffc0-4421-bf71-5fdaf07cc22b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.582567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5207350-ffc0-4421-bf71-5fdaf07cc22b admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.583205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5207350-ffc0-4421-bf71-5fdaf07cc22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.583379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb14df8-6a09-40b0-bb57-3510cec59379 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.585236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5207350-ffc0-4421-bf71-5fdaf07cc22b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.597313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 794/1594] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.597932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb14df8-6a09-40b0-bb57-3510cec59379 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.599550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ca193-38c5-4774-ab0b-e6d19d823663 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.600573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ca193-38c5-4774-ab0b-e6d19d823663 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.601389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ca193-38c5-4774-ab0b-e6d19d823663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.603606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3aa03b-63da-485b-821f-2f23f11e79d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.604271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3aa03b-63da-485b-821f-2f23f11e79d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.604548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3aa03b-63da-485b-821f-2f23f11e79d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.604821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3aa03b-63da-485b-821f-2f23f11e79d8 admin admin] PATH_INFO: `/v3/projects/552017cc73234b8db2ac15a8496e649e/users/61b9ae7499154e5eab4f3c23d8bb6785/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.623309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3aa03b-63da-485b-821f-2f23f11e79d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=552017cc73234b8db2ac15a8496e649e, user_id=61b9ae7499154e5eab4f3c23d8bb6785, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.626292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3aa03b-63da-485b-821f-2f23f11e79d8 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.637221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2b2ca193-38c5-4774-ab0b-e6d19d823663 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:52.638516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2b2ca193-38c5-4774-ab0b-e6d19d823663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:52.642204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 801/1595] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:52 2022] PUT /identity/v3/projects/552017cc73234b8db2ac15a8496e649e/users/61b9ae7499154e5eab4f3c23d8bb6785/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:52.646001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3aa03b-63da-485b-821f-2f23f11e79d8 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.691796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:52.724167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3aa03b-63da-485b-821f-2f23f11e79d8 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.724909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 795/1596] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:52.728921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ca193-38c5-4774-ab0b-e6d19d823663 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.729591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef171-cadb-4c8c-8433-cdd5c683edc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.730312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef171-cadb-4c8c-8433-cdd5c683edc7 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.730945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef171-cadb-4c8c-8433-cdd5c683edc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.731346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ef171-cadb-4c8c-8433-cdd5c683edc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.732088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef171-cadb-4c8c-8433-cdd5c683edc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.734083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ef171-cadb-4c8c-8433-cdd5c683edc7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.741157 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ca193-38c5-4774-ab0b-e6d19d823663 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.746449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 802/1597] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.747359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef171-cadb-4c8c-8433-cdd5c683edc7 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.749679 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e5516-30af-46d6-84e5-9cf2daec28c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.750442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5516-30af-46d6-84e5-9cf2daec28c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.750732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5516-30af-46d6-84e5-9cf2daec28c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.751010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e5516-30af-46d6-84e5-9cf2daec28c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.751877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e5516-30af-46d6-84e5-9cf2daec28c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.753517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e5516-30af-46d6-84e5-9cf2daec28c6 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.760740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ef171-cadb-4c8c-8433-cdd5c683edc7 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.765990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc87a4-4940-49e8-a620-a0851cc03cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.766736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc87a4-4940-49e8-a620-a0851cc03cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.767084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc87a4-4940-49e8-a620-a0851cc03cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.767429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc87a4-4940-49e8-a620-a0851cc03cbe admin admin] PATH_INFO: `/v3/projects/ae6f32eae82d4730ad4a0a5585ff4235/users/123b601ea1f24cec8604de94e2099ffe/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.770493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 796/1598] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.772815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e5516-30af-46d6-84e5-9cf2daec28c6 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.788427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc87a4-4940-49e8-a620-a0851cc03cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6f32eae82d4730ad4a0a5585ff4235, user_id=123b601ea1f24cec8604de94e2099ffe, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.789021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e5516-30af-46d6-84e5-9cf2daec28c6 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.793149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc87a4-4940-49e8-a620-a0851cc03cbe admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.794585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bcef7-0cd9-437f-b2bf-9b10e9f98acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.795438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bcef7-0cd9-437f-b2bf-9b10e9f98acc admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.795752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bcef7-0cd9-437f-b2bf-9b10e9f98acc admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.796037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95bcef7-0cd9-437f-b2bf-9b10e9f98acc admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.796771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bcef7-0cd9-437f-b2bf-9b10e9f98acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.798722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95bcef7-0cd9-437f-b2bf-9b10e9f98acc admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.813419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 797/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.818241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bcef7-0cd9-437f-b2bf-9b10e9f98acc admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.831092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 803/1600] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:52 2022] PUT /identity/v3/projects/ae6f32eae82d4730ad4a0a5585ff4235/users/123b601ea1f24cec8604de94e2099ffe/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:52.831092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d8b78-e566-44e7-a54c-849824e05651 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.831092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d8b78-e566-44e7-a54c-849824e05651 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.831092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d8b78-e566-44e7-a54c-849824e05651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.834537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95bcef7-0cd9-437f-b2bf-9b10e9f98acc admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.851478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fc3d8b78-e566-44e7-a54c-849824e05651 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:52.852816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fc3d8b78-e566-44e7-a54c-849824e05651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:52.940168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 804/1601] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:52.941433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d8b78-e566-44e7-a54c-849824e05651 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.947200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4454f-6f29-48dc-b9f9-f67732b65545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.947836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4454f-6f29-48dc-b9f9-f67732b65545 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.948116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4454f-6f29-48dc-b9f9-f67732b65545 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.948544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4454f-6f29-48dc-b9f9-f67732b65545 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:52.949205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4454f-6f29-48dc-b9f9-f67732b65545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:52.951106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4454f-6f29-48dc-b9f9-f67732b65545 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:52.966428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 798/1602] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:52.966428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4454f-6f29-48dc-b9f9-f67732b65545 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:52.978699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4454f-6f29-48dc-b9f9-f67732b65545 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:52.983519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40bdf1-f47f-4373-9daa-3124df8f14aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:52.984314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40bdf1-f47f-4373-9daa-3124df8f14aa admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:52.984597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40bdf1-f47f-4373-9daa-3124df8f14aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:52.984870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40bdf1-f47f-4373-9daa-3124df8f14aa admin admin] PATH_INFO: `/v3/projects/f03c01ff6ef24c59b65362c66ad1a1ea/users/3177776f47ef4d3e937c2e8efb360e79/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.004749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40bdf1-f47f-4373-9daa-3124df8f14aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03c01ff6ef24c59b65362c66ad1a1ea, user_id=3177776f47ef4d3e937c2e8efb360e79, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:53.010781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40bdf1-f47f-4373-9daa-3124df8f14aa admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:53.025123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d8b78-e566-44e7-a54c-849824e05651 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:53.032370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 799/1603] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:52 2022] PUT /identity/v3/projects/f03c01ff6ef24c59b65362c66ad1a1ea/users/3177776f47ef4d3e937c2e8efb360e79/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:53.035284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40bdf1-f47f-4373-9daa-3124df8f14aa admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:53.078939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:53.116668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef50d14-28aa-4d96-9f7d-c52a12e0fd38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:53.117389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef50d14-28aa-4d96-9f7d-c52a12e0fd38 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.118245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef50d14-28aa-4d96-9f7d-c52a12e0fd38 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.118525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef50d14-28aa-4d96-9f7d-c52a12e0fd38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.120280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40bdf1-f47f-4373-9daa-3124df8f14aa admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:53.125709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937fcb5-948d-4a00-a85e-863eaf641800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:53.126473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937fcb5-948d-4a00-a85e-863eaf641800 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.126857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937fcb5-948d-4a00-a85e-863eaf641800 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.127210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937fcb5-948d-4a00-a85e-863eaf641800 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.127928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937fcb5-948d-4a00-a85e-863eaf641800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:53.129520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef50d14-28aa-4d96-9f7d-c52a12e0fd38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:53.129683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937fcb5-948d-4a00-a85e-863eaf641800 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:53.131487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef50d14-28aa-4d96-9f7d-c52a12e0fd38 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:53.141466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 800/1604] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:53.142437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937fcb5-948d-4a00-a85e-863eaf641800 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:53.156253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937fcb5-948d-4a00-a85e-863eaf641800 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:53.160375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 805/1605] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:53.161508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280c20f-79f7-424a-b558-d36043378484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:53.162287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88418e-242e-458a-8c81-b31832c749e3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.162535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280c20f-79f7-424a-b558-d36043378484 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.162835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280c20f-79f7-424a-b558-d36043378484 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.163104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280c20f-79f7-424a-b558-d36043378484 admin admin] PATH_INFO: `/v3/projects/da1c84a716c84e639121e0223e547df0/users/baa16752837348d392424acb21d8ad69/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.165840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88418e-242e-458a-8c81-b31832c749e3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.166318 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88418e-242e-458a-8c81-b31832c749e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.189596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:53.199148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280c20f-79f7-424a-b558-d36043378484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da1c84a716c84e639121e0223e547df0, user_id=baa16752837348d392424acb21d8ad69, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:53.208143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2280c20f-79f7-424a-b558-d36043378484 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:53.210179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9d88418e-242e-458a-8c81-b31832c749e3 None None] MFA Rules not processed for user `123b601ea1f24cec8604de94e2099ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:53.211981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9d88418e-242e-458a-8c81-b31832c749e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:53.240155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 801/1606] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:53 2022] PUT /identity/v3/projects/da1c84a716c84e639121e0223e547df0/users/baa16752837348d392424acb21d8ad69/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:53.243954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280c20f-79f7-424a-b558-d36043378484 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:53.301734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 806/1607] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:53.306169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88418e-242e-458a-8c81-b31832c749e3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:53.376894 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88418e-242e-458a-8c81-b31832c749e3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:53.382174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280c20f-79f7-424a-b558-d36043378484 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:53.382174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83d247-9f29-4f89-a02a-0bbd1531e592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:53.382174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83d247-9f29-4f89-a02a-0bbd1531e592 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.382174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83d247-9f29-4f89-a02a-0bbd1531e592 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.382850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac83d247-9f29-4f89-a02a-0bbd1531e592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.462385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4b8fe8-8122-4274-a2ff-4842a662f4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:53.463077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b8fe8-8122-4274-a2ff-4842a662f4f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.463463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b8fe8-8122-4274-a2ff-4842a662f4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.463740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b8fe8-8122-4274-a2ff-4842a662f4f0 admin admin] PATH_INFO: `/v3/projects/2981eddb8742492fac516851f761e2f9/users/15069aa932aa473983c4102961b855d5/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.483900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b8fe8-8122-4274-a2ff-4842a662f4f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2981eddb8742492fac516851f761e2f9, user_id=15069aa932aa473983c4102961b855d5, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:53.487404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4b8fe8-8122-4274-a2ff-4842a662f4f0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:53.510290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 807/1608] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:53 2022] PUT /identity/v3/projects/2981eddb8742492fac516851f761e2f9/users/15069aa932aa473983c4102961b855d5/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:53.512439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0875a31-f507-405c-9f5c-0436454c901b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.513010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0875a31-f507-405c-9f5c-0436454c901b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.513010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0875a31-f507-405c-9f5c-0436454c901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.520059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83d247-9f29-4f89-a02a-0bbd1531e592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:53.521675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac83d247-9f29-4f89-a02a-0bbd1531e592 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:53.547402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 802/1609] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:53.548278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83d247-9f29-4f89-a02a-0bbd1531e592 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:53.557314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e0875a31-f507-405c-9f5c-0436454c901b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:53.558643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e0875a31-f507-405c-9f5c-0436454c901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:53.561977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac83d247-9f29-4f89-a02a-0bbd1531e592 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:53.630739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:53.644334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1d6ed-7c28-4bf7-ac87-1909ebe8471e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:53.645322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d6ed-7c28-4bf7-ac87-1909ebe8471e admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.645734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d6ed-7c28-4bf7-ac87-1909ebe8471e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.646020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d6ed-7c28-4bf7-ac87-1909ebe8471e admin admin] PATH_INFO: `/v3/projects/c6b976c5c00245fabdb8577b7c182e20/users/805b4161174b4d47b85153ad41dc4eee/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.651068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 808/1610] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:53.653210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f14c70-b737-4e2a-bf4f-7d14c3faee22 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.653407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f14c70-b737-4e2a-bf4f-7d14c3faee22 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.653693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f14c70-b737-4e2a-bf4f-7d14c3faee22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.670570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1d6ed-7c28-4bf7-ac87-1909ebe8471e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b976c5c00245fabdb8577b7c182e20, user_id=805b4161174b4d47b85153ad41dc4eee, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:53.673528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1d6ed-7c28-4bf7-ac87-1909ebe8471e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:53.697798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-37f14c70-b737-4e2a-bf4f-7d14c3faee22 None None] MFA Rules not processed for user `baa16752837348d392424acb21d8ad69`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:53.700117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-37f14c70-b737-4e2a-bf4f-7d14c3faee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:53.709466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 803/1611] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:53 2022] PUT /identity/v3/projects/c6b976c5c00245fabdb8577b7c182e20/users/805b4161174b4d47b85153ad41dc4eee/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:53.714810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1d6ed-7c28-4bf7-ac87-1909ebe8471e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:53.724261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:53.836029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1d6ed-7c28-4bf7-ac87-1909ebe8471e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:53.836280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 809/1612] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:53.842083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8a45e-3a3e-4b2f-8c43-382bd571ec36 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.842445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8a45e-3a3e-4b2f-8c43-382bd571ec36 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.842731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8a45e-3a3e-4b2f-8c43-382bd571ec36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.846775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6a2bb-ba4b-44d1-a4b4-54b1aaa4f346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:53.847777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a2bb-ba4b-44d1-a4b4-54b1aaa4f346 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.848723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a2bb-ba4b-44d1-a4b4-54b1aaa4f346 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.849324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a2bb-ba4b-44d1-a4b4-54b1aaa4f346 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.850254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6a2bb-ba4b-44d1-a4b4-54b1aaa4f346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:53.855330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6a2bb-ba4b-44d1-a4b4-54b1aaa4f346 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:53.867247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-2de8a45e-3a3e-4b2f-8c43-382bd571ec36 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:53.868527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2de8a45e-3a3e-4b2f-8c43-382bd571ec36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:53.884569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 810/1613] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:53.884569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8a45e-3a3e-4b2f-8c43-382bd571ec36 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:53.899576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 804/1614] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:53 2022] POST /identity/v3/projects => generated 357 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:53.901892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f92722-8bc6-40ac-9042-e0703ef466ec None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:53.902209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f92722-8bc6-40ac-9042-e0703ef466ec None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:53.902737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f92722-8bc6-40ac-9042-e0703ef466ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:53.941724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-71f92722-8bc6-40ac-9042-e0703ef466ec None None] MFA Rules not processed for user `805b4161174b4d47b85153ad41dc4eee`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:53.943102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-71f92722-8bc6-40ac-9042-e0703ef466ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:53.975019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8a45e-3a3e-4b2f-8c43-382bd571ec36 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.079785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc04092-f29e-43e4-879d-81dc83670b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.080615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc04092-f29e-43e4-879d-81dc83670b14 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.080989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc04092-f29e-43e4-879d-81dc83670b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.081403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc04092-f29e-43e4-879d-81dc83670b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.101198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 805/1615] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:54.101198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f92722-8bc6-40ac-9042-e0703ef466ec None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.185148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc04092-f29e-43e4-879d-81dc83670b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.188119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc04092-f29e-43e4-879d-81dc83670b14 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.209975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 811/1616] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:54.212301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc04092-f29e-43e4-879d-81dc83670b14 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.225368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc04092-f29e-43e4-879d-81dc83670b14 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.233054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:54.236206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0957bd0-3724-4d09-aec8-3fcd44e08c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.237069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0957bd0-3724-4d09-aec8-3fcd44e08c43 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.238410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0957bd0-3724-4d09-aec8-3fcd44e08c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.238702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0957bd0-3724-4d09-aec8-3fcd44e08c43 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.247789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0957bd0-3724-4d09-aec8-3fcd44e08c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.250022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0957bd0-3724-4d09-aec8-3fcd44e08c43 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.259496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f92722-8bc6-40ac-9042-e0703ef466ec None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.260231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:54.276417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cb9ed-853a-4a8c-a742-c90afdb5f634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.277069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cb9ed-853a-4a8c-a742-c90afdb5f634 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.277357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cb9ed-853a-4a8c-a742-c90afdb5f634 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.277711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cb9ed-853a-4a8c-a742-c90afdb5f634 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.278373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cb9ed-853a-4a8c-a742-c90afdb5f634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.283866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cb9ed-853a-4a8c-a742-c90afdb5f634 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.305427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 812/1617] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:54 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:54.306423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0957bd0-3724-4d09-aec8-3fcd44e08c43 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.319564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0957bd0-3724-4d09-aec8-3fcd44e08c43 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.324627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000fb58-5ab8-4df5-b71c-a5d6bc57dfe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.325391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000fb58-5ab8-4df5-b71c-a5d6bc57dfe2 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.325735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000fb58-5ab8-4df5-b71c-a5d6bc57dfe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.326076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000fb58-5ab8-4df5-b71c-a5d6bc57dfe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.338994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000fb58-5ab8-4df5-b71c-a5d6bc57dfe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.340688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000fb58-5ab8-4df5-b71c-a5d6bc57dfe2 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.355852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 813/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:54.357404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9719-1629-4979-b3f9-4f85724a86a0 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.357845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9719-1629-4979-b3f9-4f85724a86a0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.358150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e9719-1629-4979-b3f9-4f85724a86a0 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.359471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 814/1619] 10.222.0.23 () {54 vars in 962 bytes} [Tue Sep 13 19:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:54.366567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9719-1629-4979-b3f9-4f85724a86a0 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.384618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 806/1620] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:48:54 2022] POST /identity/v3/projects => generated 363 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:54.384618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa8e3a-e9a2-4706-bbbe-f8e8d3abdf48 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.384618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa8e3a-e9a2-4706-bbbe-f8e8d3abdf48 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.384618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa8e3a-e9a2-4706-bbbe-f8e8d3abdf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.385830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e9719-1629-4979-b3f9-4f85724a86a0 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.492192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-d5fa8e3a-e9a2-4706-bbbe-f8e8d3abdf48 None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:54.494033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d5fa8e3a-e9a2-4706-bbbe-f8e8d3abdf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:54.524571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a18734-98d5-477f-ba82-846475fcbe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.525215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18734-98d5-477f-ba82-846475fcbe0a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.525494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18734-98d5-477f-ba82-846475fcbe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.525815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a18734-98d5-477f-ba82-846475fcbe0a admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.526461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a18734-98d5-477f-ba82-846475fcbe0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.531279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a18734-98d5-477f-ba82-846475fcbe0a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.544915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 815/1621] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:54.546162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a18734-98d5-477f-ba82-846475fcbe0a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.561437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a18734-98d5-477f-ba82-846475fcbe0a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.567737 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883b0ff-e8f0-4b95-9348-9ac4ff3e7601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.569298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883b0ff-e8f0-4b95-9348-9ac4ff3e7601 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.569717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883b0ff-e8f0-4b95-9348-9ac4ff3e7601 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.570094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4883b0ff-e8f0-4b95-9348-9ac4ff3e7601 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.571751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883b0ff-e8f0-4b95-9348-9ac4ff3e7601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.573647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4883b0ff-e8f0-4b95-9348-9ac4ff3e7601 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.614123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 816/1622] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:48:54 2022] POST /identity/v3/users => generated 288 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:48:54.618949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883b0ff-e8f0-4b95-9348-9ac4ff3e7601 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.633243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4883b0ff-e8f0-4b95-9348-9ac4ff3e7601 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.643140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60382-da47-47cb-8aa7-3a875a91414e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.644153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60382-da47-47cb-8aa7-3a875a91414e admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.644643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60382-da47-47cb-8aa7-3a875a91414e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.645124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60382-da47-47cb-8aa7-3a875a91414e admin admin] PATH_INFO: `/v3/projects/f5fce0c227ec4bd08c780f97a2297597/users/eada5dcab7134f2cbdf74e924c59db94/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.645548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 807/1623] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Sep 13 19:48:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:54.646537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa8e3a-e9a2-4706-bbbe-f8e8d3abdf48 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.669749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa8e3a-e9a2-4706-bbbe-f8e8d3abdf48 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.682756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e60382-da47-47cb-8aa7-3a875a91414e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5fce0c227ec4bd08c780f97a2297597, user_id=eada5dcab7134f2cbdf74e924c59db94, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.685113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5d3ed-6053-43bf-99cc-a1024991a1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.685784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5d3ed-6053-43bf-99cc-a1024991a1da admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.686072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5d3ed-6053-43bf-99cc-a1024991a1da admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.686349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5d3ed-6053-43bf-99cc-a1024991a1da admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.687009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5d3ed-6053-43bf-99cc-a1024991a1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.688628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5d3ed-6053-43bf-99cc-a1024991a1da admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.688943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e60382-da47-47cb-8aa7-3a875a91414e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.712777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 808/1624] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:54.713700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5d3ed-6053-43bf-99cc-a1024991a1da admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.728488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 817/1625] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:54 2022] PUT /identity/v3/projects/f5fce0c227ec4bd08c780f97a2297597/users/eada5dcab7134f2cbdf74e924c59db94/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:54.728488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60382-da47-47cb-8aa7-3a875a91414e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.749100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5d3ed-6053-43bf-99cc-a1024991a1da admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.864414 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60382-da47-47cb-8aa7-3a875a91414e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:54.880230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895382-ab5a-4d50-bce6-45b516c631a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.881038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895382-ab5a-4d50-bce6-45b516c631a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.881783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895382-ab5a-4d50-bce6-45b516c631a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.882141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d895382-ab5a-4d50-bce6-45b516c631a2 admin admin] PATH_INFO: `/v3/projects/e6a3016b765d4ef0be5fba3e9b789c84/users/902693d2965e41348e53f77277cd03f8/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.895897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7efa5-d32b-445f-8063-ce9411ef8552 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:54.896847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7efa5-d32b-445f-8063-ce9411ef8552 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:54.896847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7efa5-d32b-445f-8063-ce9411ef8552 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:54.897081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7efa5-d32b-445f-8063-ce9411ef8552 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:54.904807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d895382-ab5a-4d50-bce6-45b516c631a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a3016b765d4ef0be5fba3e9b789c84, user_id=902693d2965e41348e53f77277cd03f8, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:54.909830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d895382-ab5a-4d50-bce6-45b516c631a2 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:54.935172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 818/1626] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:54 2022] PUT /identity/v3/projects/e6a3016b765d4ef0be5fba3e9b789c84/users/902693d2965e41348e53f77277cd03f8/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:54.938897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895382-ab5a-4d50-bce6-45b516c631a2 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:54.979332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:55.000422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:55.059654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7efa5-d32b-445f-8063-ce9411ef8552 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:55.064408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7efa5-d32b-445f-8063-ce9411ef8552 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:55.095423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 809/1627] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:55.098324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7efa5-d32b-445f-8063-ce9411ef8552 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:55.113824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d895382-ab5a-4d50-bce6-45b516c631a2 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:55.121081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7efa5-d32b-445f-8063-ce9411ef8552 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:55.125421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea41ed-df7b-42f1-86ae-6cc6d21b293c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:55.125824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea41ed-df7b-42f1-86ae-6cc6d21b293c admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.126106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea41ed-df7b-42f1-86ae-6cc6d21b293c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.126421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea41ed-df7b-42f1-86ae-6cc6d21b293c admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.127087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea41ed-df7b-42f1-86ae-6cc6d21b293c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:55.129471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea41ed-df7b-42f1-86ae-6cc6d21b293c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:55.134045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147a5da-316d-4f83-89cc-536ca9e25623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:55.135503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147a5da-316d-4f83-89cc-536ca9e25623 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.135790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147a5da-316d-4f83-89cc-536ca9e25623 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.136068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147a5da-316d-4f83-89cc-536ca9e25623 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.139671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147a5da-316d-4f83-89cc-536ca9e25623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:55.141491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147a5da-316d-4f83-89cc-536ca9e25623 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:55.162229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 819/1628] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:55.168073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea41ed-df7b-42f1-86ae-6cc6d21b293c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:55.187986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 810/1629] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:48:55.193906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147a5da-316d-4f83-89cc-536ca9e25623 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:55.207577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea41ed-df7b-42f1-86ae-6cc6d21b293c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:55.214553 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147a5da-316d-4f83-89cc-536ca9e25623 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:55.220496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f3c55-8441-4d70-9114-7c7c9f4605e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:55.221580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f3c55-8441-4d70-9114-7c7c9f4605e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.222347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f3c55-8441-4d70-9114-7c7c9f4605e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.222625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f3c55-8441-4d70-9114-7c7c9f4605e4 admin admin] PATH_INFO: `/v3/projects/f5fce0c227ec4bd08c780f97a2297597/users/eada5dcab7134f2cbdf74e924c59db94/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.225889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f3c55-8441-4d70-9114-7c7c9f4605e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5fce0c227ec4bd08c780f97a2297597, user_id=eada5dcab7134f2cbdf74e924c59db94, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:55.229500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f3c55-8441-4d70-9114-7c7c9f4605e4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:55.233324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd59c77-7bad-4101-9b2e-1e664bfb62de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:55.234083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59c77-7bad-4101-9b2e-1e664bfb62de admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.234366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59c77-7bad-4101-9b2e-1e664bfb62de admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.234643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd59c77-7bad-4101-9b2e-1e664bfb62de admin admin] PATH_INFO: `/v3/projects/e6a3016b765d4ef0be5fba3e9b789c84/users/902693d2965e41348e53f77277cd03f8/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.238147 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd59c77-7bad-4101-9b2e-1e664bfb62de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6a3016b765d4ef0be5fba3e9b789c84, user_id=902693d2965e41348e53f77277cd03f8, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:55.240868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd59c77-7bad-4101-9b2e-1e664bfb62de admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:55.262566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 820/1630] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:55 2022] PUT /identity/v3/projects/f5fce0c227ec4bd08c780f97a2297597/users/eada5dcab7134f2cbdf74e924c59db94/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:55.275557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1915a1e-04c6-4452-852a-e1fedd6fbf4e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.275901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1915a1e-04c6-4452-852a-e1fedd6fbf4e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.276113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1915a1e-04c6-4452-852a-e1fedd6fbf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.287769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 811/1631] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:48:55 2022] PUT /identity/v3/projects/e6a3016b765d4ef0be5fba3e9b789c84/users/902693d2965e41348e53f77277cd03f8/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:55.291082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924fd34-b6e3-45ec-b2bd-8a5122ba8c39 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.291374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924fd34-b6e3-45ec-b2bd-8a5122ba8c39 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.291928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924fd34-b6e3-45ec-b2bd-8a5122ba8c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.339172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2924fd34-b6e3-45ec-b2bd-8a5122ba8c39 None None] MFA Rules not processed for user `902693d2965e41348e53f77277cd03f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:55.341523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2924fd34-b6e3-45ec-b2bd-8a5122ba8c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:55.357388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e1915a1e-04c6-4452-852a-e1fedd6fbf4e None None] MFA Rules not processed for user `eada5dcab7134f2cbdf74e924c59db94`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:55.359929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e1915a1e-04c6-4452-852a-e1fedd6fbf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:55.461070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 812/1632] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:55.472595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924fd34-b6e3-45ec-b2bd-8a5122ba8c39 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:55.538700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 821/1633] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:55.551954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1915a1e-04c6-4452-852a-e1fedd6fbf4e None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:55.631873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924fd34-b6e3-45ec-b2bd-8a5122ba8c39 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:55.639396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897a046-138e-4c5e-b36b-8a3c619a0d7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:55.641363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897a046-138e-4c5e-b36b-8a3c619a0d7d service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.642014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897a046-138e-4c5e-b36b-8a3c619a0d7d service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.643413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897a046-138e-4c5e-b36b-8a3c619a0d7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.673645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897a046-138e-4c5e-b36b-8a3c619a0d7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:55.682309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897a046-138e-4c5e-b36b-8a3c619a0d7d service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:55.722840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 813/1634] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:48:55.727730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8178b-87ee-4fd3-aebf-c01d812af8d7 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.729520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8178b-87ee-4fd3-aebf-c01d812af8d7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.732184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8178b-87ee-4fd3-aebf-c01d812af8d7 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.736512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1915a1e-04c6-4452-852a-e1fedd6fbf4e None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:55.739884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 814/1635] 10.222.0.23 () {54 vars in 912 bytes} [Tue Sep 13 19:48:55 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:55.745252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ea76c-544a-4793-a82e-89788be34c60 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.745871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ea76c-544a-4793-a82e-89788be34c60 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.746374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ea76c-544a-4793-a82e-89788be34c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.757587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc479dc-8d06-4ae9-9ae6-b7376aa4ecf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:55.759233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc479dc-8d06-4ae9-9ae6-b7376aa4ecf9 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:55.760085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc479dc-8d06-4ae9-9ae6-b7376aa4ecf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:55.761051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc479dc-8d06-4ae9-9ae6-b7376aa4ecf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:55.784659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc479dc-8d06-4ae9-9ae6-b7376aa4ecf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:55.786740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc479dc-8d06-4ae9-9ae6-b7376aa4ecf9 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:55.796311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-125ea76c-544a-4793-a82e-89788be34c60 None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:55.798270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-125ea76c-544a-4793-a82e-89788be34c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:55.812526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 822/1636] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:55.894766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:56.005508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 815/1637] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:48:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:56.036614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc479dc-8d06-4ae9-9ae6-b7376aa4ecf9 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:56.047969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc479dc-8d06-4ae9-9ae6-b7376aa4ecf9 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:56.058257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab312408-3e6c-4490-9b92-4e5642b25877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:56.060087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab312408-3e6c-4490-9b92-4e5642b25877 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:56.060615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab312408-3e6c-4490-9b92-4e5642b25877 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:56.060987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab312408-3e6c-4490-9b92-4e5642b25877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:56.118562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ea76c-544a-4793-a82e-89788be34c60 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:56.145475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ea76c-544a-4793-a82e-89788be34c60 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:56.146934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8750e86-686d-427d-afcb-c2258b66e9cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:56.149965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8750e86-686d-427d-afcb-c2258b66e9cd service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:56.151193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8750e86-686d-427d-afcb-c2258b66e9cd service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:56.152015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8750e86-686d-427d-afcb-c2258b66e9cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:56.154084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:56.197418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8750e86-686d-427d-afcb-c2258b66e9cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:56.201563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8750e86-686d-427d-afcb-c2258b66e9cd service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:56.251032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 816/1638] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:48:56.257309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab312408-3e6c-4490-9b92-4e5642b25877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:56.260292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab312408-3e6c-4490-9b92-4e5642b25877 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:56.286873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 823/1639] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:56 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:57.446501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8750e86-686d-427d-afcb-c2258b66e9cd service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:57.573881 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8750e86-686d-427d-afcb-c2258b66e9cd service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:57.586278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:57.587510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:57.589949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:57.590352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] PATH_INFO: `/v3/users/baa16752837348d392424acb21d8ad69` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:57.591910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa16752837348d392424acb21d8ad69)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:57.594842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: WARNING py.warnings [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:48:57.594842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: warnings.warn(msg) Sep 13 19:48:57.594842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: Sep 13 19:48:57.598833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:57.738972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa16752837348d392424acb21d8ad69'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:48:57.755271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa16752837348d392424acb21d8ad69'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:48:57.756092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa16752837348d392424acb21d8ad69'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:48:57.795197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa16752837348d392424acb21d8ad69'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:48:57.823286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-54e4dab3-1632-4f00-b53f-0a50a6d03e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa16752837348d392424acb21d8ad69'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:48:57.840318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 817/1640] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:48:57 2022] DELETE /identity/v3/users/baa16752837348d392424acb21d8ad69 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:48:57.844824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfef04-2663-43c5-a406-cccee412a523 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:57.845847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfef04-2663-43c5-a406-cccee412a523 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:57.846327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfef04-2663-43c5-a406-cccee412a523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:57.902665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ebbfef04-2663-43c5-a406-cccee412a523 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:57.907512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ebbfef04-2663-43c5-a406-cccee412a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:58.074927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 824/1641] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:58.166402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777bb12-1eb6-4c1e-9d58-873fef1eb651 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:58.166402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777bb12-1eb6-4c1e-9d58-873fef1eb651 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:58.166402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777bb12-1eb6-4c1e-9d58-873fef1eb651 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:58.166402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 818/1642] 10.222.0.23 () {54 vars in 962 bytes} [Tue Sep 13 19:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:58.188669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2859fe-f9bd-4867-be3f-b7833bde8e37 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:58.189122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2859fe-f9bd-4867-be3f-b7833bde8e37 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:58.189564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2859fe-f9bd-4867-be3f-b7833bde8e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:58.246156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-7b2859fe-f9bd-4867-be3f-b7833bde8e37 None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:58.253219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7b2859fe-f9bd-4867-be3f-b7833bde8e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:58.319307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777bb12-1eb6-4c1e-9d58-873fef1eb651 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:58.322493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6777bb12-1eb6-4c1e-9d58-873fef1eb651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:58.437867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:58.442675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 825/1643] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Sep 13 19:48:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:58.452790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328901-07fa-4586-b0dd-056c81e9da53 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:58.453094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328901-07fa-4586-b0dd-056c81e9da53 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:58.453353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33328901-07fa-4586-b0dd-056c81e9da53 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:58.454789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 826/1644] 10.222.0.23 () {54 vars in 912 bytes} [Tue Sep 13 19:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:58.465234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfca44-6f24-47d6-8615-f1bd7edab99c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:58.465234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfca44-6f24-47d6-8615-f1bd7edab99c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:58.465493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfca44-6f24-47d6-8615-f1bd7edab99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:58.468468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777bb12-1eb6-4c1e-9d58-873fef1eb651 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:58.479183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117ee81-21c7-4cc3-90a5-6bacb1f3a3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:58.482544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117ee81-21c7-4cc3-90a5-6bacb1f3a3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:58.482986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117ee81-21c7-4cc3-90a5-6bacb1f3a3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:58.483528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117ee81-21c7-4cc3-90a5-6bacb1f3a3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:58.507135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f0dfca44-6f24-47d6-8615-f1bd7edab99c None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:58.508648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f0dfca44-6f24-47d6-8615-f1bd7edab99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:58.515622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117ee81-21c7-4cc3-90a5-6bacb1f3a3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:58.517754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117ee81-21c7-4cc3-90a5-6bacb1f3a3f0 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:58.533893 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 827/1645] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:48:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:58.540293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 819/1646] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:48:58.548241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfca44-6f24-47d6-8615-f1bd7edab99c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:58.549810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f0dfca44-6f24-47d6-8615-f1bd7edab99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:58.637374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfca44-6f24-47d6-8615-f1bd7edab99c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:58.644381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8fe525-4e87-445c-9e98-73c22cc8f577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:58.645019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fe525-4e87-445c-9e98-73c22cc8f577 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:58.645288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fe525-4e87-445c-9e98-73c22cc8f577 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:58.645560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8fe525-4e87-445c-9e98-73c22cc8f577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:58.655155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:58.661219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fe525-4e87-445c-9e98-73c22cc8f577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:58.663085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8fe525-4e87-445c-9e98-73c22cc8f577 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:58.709741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 828/1647] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:59.021446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6acd-af92-494b-8628-09b7ae7d2d5e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:59.022651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6acd-af92-494b-8628-09b7ae7d2d5e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:59.024095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb6acd-af92-494b-8628-09b7ae7d2d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:59.113062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-6cfb6acd-af92-494b-8628-09b7ae7d2d5e None None] MFA Rules not processed for user `a0e295fda2af4157b196c6130aa2d6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:59.114806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6cfb6acd-af92-494b-8628-09b7ae7d2d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:59.267957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 820/1648] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:59.280036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b2ef0-f1c0-45cd-acae-b5766346011e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:59.281009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b2ef0-f1c0-45cd-acae-b5766346011e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:59.281470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b2ef0-f1c0-45cd-acae-b5766346011e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:59.328464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a89b2ef0-f1c0-45cd-acae-b5766346011e None None] MFA Rules not processed for user `a0e295fda2af4157b196c6130aa2d6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:59.330354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a89b2ef0-f1c0-45cd-acae-b5766346011e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:59.344099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 829/1649] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:48:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:59.357735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb6acd-af92-494b-8628-09b7ae7d2d5e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:59.389864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb6acd-af92-494b-8628-09b7ae7d2d5e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:59.403629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:59.404444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:59.404850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:59.405230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:59.426169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:59.427994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:59.458359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 821/1650] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:00.015061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b2ef0-f1c0-45cd-acae-b5766346011e None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:00.017024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a89b2ef0-f1c0-45cd-acae-b5766346011e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:00.046194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b2ef0-f1c0-45cd-acae-b5766346011e None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:00.060780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:00.062361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:00.062650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:00.062929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] PATH_INFO: `/v3/projects/da1c84a716c84e639121e0223e547df0` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:00.066325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da1c84a716c84e639121e0223e547df0)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:00.069080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: WARNING py.warnings [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:49:00.069080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: warnings.warn(msg) Sep 13 19:49:00.069080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: Sep 13 19:49:00.074589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:00.256822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da1c84a716c84e639121e0223e547df0', 'request_id': 'req-7e69a256-689d-4006-973a-7c1412ff366c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.310164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da1c84a716c84e639121e0223e547df0', 'request_id': 'req-7e69a256-689d-4006-973a-7c1412ff366c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.378394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:00.380398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:00.391632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da1c84a716c84e639121e0223e547df0', 'request_id': 'req-7e69a256-689d-4006-973a-7c1412ff366c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.405410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776632c3-7086-4c45-8adf-b7e11e838d00 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:00.418883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] The token cache is being invalidate because project da1c84a716c84e639121e0223e547df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:00.419219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.420340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.429686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 830/1651] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:00 2022] DELETE /identity/v3/projects/da1c84a716c84e639121e0223e547df0 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:00.435187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:00.436714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:00.585746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:00.587816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:00.589269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:00.589451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] PATH_INFO: `/v3/users/805b4161174b4d47b85153ad41dc4eee` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:00.594716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805b4161174b4d47b85153ad41dc4eee)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:00.603131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:00.604445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e69a256-689d-4006-973a-7c1412ff366c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:00.615770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:00.616817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:00.617986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:00.618538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] PATH_INFO: `/v3/users/902693d2965e41348e53f77277cd03f8` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:00.623885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902693d2965e41348e53f77277cd03f8)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:00.627188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:00.728425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805b4161174b4d47b85153ad41dc4eee'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.735375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902693d2965e41348e53f77277cd03f8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.745295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805b4161174b4d47b85153ad41dc4eee'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.746971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902693d2965e41348e53f77277cd03f8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.747513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805b4161174b4d47b85153ad41dc4eee'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.747672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902693d2965e41348e53f77277cd03f8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.762896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902693d2965e41348e53f77277cd03f8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.763903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:00.764763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:00.773270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6b83994-bf8c-454f-a54c-0dda40d75ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902693d2965e41348e53f77277cd03f8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.778042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805b4161174b4d47b85153ad41dc4eee'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.785748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 831/1652] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:00 2022] DELETE /identity/v3/users/902693d2965e41348e53f77277cd03f8 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:00.796055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a255196-5fc0-4455-9b54-8916842132d0 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:00.797737 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a255196-5fc0-4455-9b54-8916842132d0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:00.798168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a255196-5fc0-4455-9b54-8916842132d0 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:00.802680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f766db0f-f83e-4fbd-96ff-25fe3d761494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805b4161174b4d47b85153ad41dc4eee'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:00.804094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 832/1653] 10.222.0.23 () {54 vars in 912 bytes} [Tue Sep 13 19:49:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:00.808684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027df184-d00d-4106-9c94-3ef638c94c8c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:00.809182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027df184-d00d-4106-9c94-3ef638c94c8c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:00.809595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027df184-d00d-4106-9c94-3ef638c94c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:00.820254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 822/1654] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:00 2022] DELETE /identity/v3/users/805b4161174b4d47b85153ad41dc4eee => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:00.823261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61bc6-e637-4fc1-9adc-efd3932eb3c7 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:00.823685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61bc6-e637-4fc1-9adc-efd3932eb3c7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:00.824321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61bc6-e637-4fc1-9adc-efd3932eb3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:00.871115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-43f61bc6-e637-4fc1-9adc-efd3932eb3c7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:00.875765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-43f61bc6-e637-4fc1-9adc-efd3932eb3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:00.930788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-027df184-d00d-4106-9c94-3ef638c94c8c None None] MFA Rules not processed for user `9f77e57b2d96461c88e2fc33db22e22d`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:00.934349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-027df184-d00d-4106-9c94-3ef638c94c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:01.026190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 823/1655] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:01.047140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f61bc6-e637-4fc1-9adc-efd3932eb3c7 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:01.049484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-43f61bc6-e637-4fc1-9adc-efd3932eb3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:01.066577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 833/1656] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:49:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:01.073116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f61bc6-e637-4fc1-9adc-efd3932eb3c7 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:01.077281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:01.078069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:01.078705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:01.079092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:01.090695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:01.092512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:01.107922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 824/1657] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:01.125382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027df184-d00d-4106-9c94-3ef638c94c8c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:01.145820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027df184-d00d-4106-9c94-3ef638c94c8c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:01.152309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f84d31-768e-46c3-bf48-c704edd7e8bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:01.153098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f84d31-768e-46c3-bf48-c704edd7e8bf service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:01.153487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f84d31-768e-46c3-bf48-c704edd7e8bf service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:01.176734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f84d31-768e-46c3-bf48-c704edd7e8bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:01.189528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f84d31-768e-46c3-bf48-c704edd7e8bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:01.191251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f84d31-768e-46c3-bf48-c704edd7e8bf service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:01.228310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 834/1658] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:01.993595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:01.995281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:02.021227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b4d18-ddb3-4d17-bd8e-25083b1a1952 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:02.035456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:02.036128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:02.036411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:02.036687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] PATH_INFO: `/v3/projects/e6a3016b765d4ef0be5fba3e9b789c84` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:02.040225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6a3016b765d4ef0be5fba3e9b789c84)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:02.042895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:02.229078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6a3016b765d4ef0be5fba3e9b789c84', 'request_id': 'req-016c0eff-136d-498a-ba73-62dc06383733'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:02.250760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6a3016b765d4ef0be5fba3e9b789c84', 'request_id': 'req-016c0eff-136d-498a-ba73-62dc06383733'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:02.272954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6a3016b765d4ef0be5fba3e9b789c84', 'request_id': 'req-016c0eff-136d-498a-ba73-62dc06383733'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:02.317697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] The token cache is being invalidate because project e6a3016b765d4ef0be5fba3e9b789c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:02.318006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:02.318710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-016c0eff-136d-498a-ba73-62dc06383733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:02.324355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 825/1659] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:01 2022] DELETE /identity/v3/projects/e6a3016b765d4ef0be5fba3e9b789c84 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:02.326117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c3c27-60fd-4ead-86db-c56811340393 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:02.326613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c3c27-60fd-4ead-86db-c56811340393 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:02.326701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c3c27-60fd-4ead-86db-c56811340393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:02.393092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ab6c3c27-60fd-4ead-86db-c56811340393 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:02.394969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ab6c3c27-60fd-4ead-86db-c56811340393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:02.657592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 835/1660] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:02.667611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea46ff3-f94b-4681-86c7-09a2e1e93fb4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:02.667910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea46ff3-f94b-4681-86c7-09a2e1e93fb4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:02.668194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea46ff3-f94b-4681-86c7-09a2e1e93fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:02.735050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8ea46ff3-f94b-4681-86c7-09a2e1e93fb4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:02.741402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8ea46ff3-f94b-4681-86c7-09a2e1e93fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:02.757298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 826/1661] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:02.760087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c3c27-60fd-4ead-86db-c56811340393 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:02.785149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c3c27-60fd-4ead-86db-c56811340393 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:02.805928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:02.806605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:02.806849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:02.807273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:02.807950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:02.808257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:02.810398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:02.853956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 836/1662] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:02.858067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea46ff3-f94b-4681-86c7-09a2e1e93fb4 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:02.869587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea46ff3-f94b-4681-86c7-09a2e1e93fb4 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:02.887964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b410be9-6e45-43d4-ab51-fedda9062a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:02.888543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b410be9-6e45-43d4-ab51-fedda9062a44 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:02.888827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b410be9-6e45-43d4-ab51-fedda9062a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:02.890136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b410be9-6e45-43d4-ab51-fedda9062a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:02.890858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b410be9-6e45-43d4-ab51-fedda9062a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:02.903731 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b410be9-6e45-43d4-ab51-fedda9062a44 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:03.002024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 827/1663] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:02 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:03.006987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:03.038874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608619f1-75e6-41a4-a538-d4773b559753 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:03.177689 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33e802-55b3-4d80-bf2b-ef40be212609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:03.178353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33e802-55b3-4d80-bf2b-ef40be212609 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.178636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33e802-55b3-4d80-bf2b-ef40be212609 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.178910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33e802-55b3-4d80-bf2b-ef40be212609 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.179635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33e802-55b3-4d80-bf2b-ef40be212609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:03.181585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33e802-55b3-4d80-bf2b-ef40be212609 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:03.199257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:03.219908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 837/1664] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:03 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:03.225392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b410be9-6e45-43d4-ab51-fedda9062a44 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:03.240125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b410be9-6e45-43d4-ab51-fedda9062a44 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:03.256076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:03.256812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.257001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.257257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.258099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:03.259875 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:03.285951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 828/1665] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:03.288603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33e802-55b3-4d80-bf2b-ef40be212609 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:03.310183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33e802-55b3-4d80-bf2b-ef40be212609 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:03.326499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0065-c70d-464c-8f4c-685c8b132037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:03.327268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0065-c70d-464c-8f4c-685c8b132037 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.327554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0065-c70d-464c-8f4c-685c8b132037 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.327831 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b0065-c70d-464c-8f4c-685c8b132037 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.328557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b0065-c70d-464c-8f4c-685c8b132037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:03.330607 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:03.333121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:03.338683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b0065-c70d-464c-8f4c-685c8b132037 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:03.364602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be8180-54c2-4146-aa80-756d8c1008cc admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:03.397509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:03.398957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.399134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.399388 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] PATH_INFO: `/v3/projects/c6b976c5c00245fabdb8577b7c182e20` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.400734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b976c5c00245fabdb8577b7c182e20)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:03.402184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 838/1666] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:03.406889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0065-c70d-464c-8f4c-685c8b132037 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:03.413482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:03.424004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:03.432384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b0065-c70d-464c-8f4c-685c8b132037 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:03.449423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:03.450961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.451272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.451537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] PATH_INFO: `/v3/projects/04b76c8ca55e44b3952d4d01a6916776/users/68c072ebb2c44d259e833f6cfa148807/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.478614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b76c8ca55e44b3952d4d01a6916776, user_id=68c072ebb2c44d259e833f6cfa148807, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:03.481884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:03.495512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b976c5c00245fabdb8577b7c182e20', 'request_id': 'req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:03.503399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b976c5c00245fabdb8577b7c182e20', 'request_id': 'req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:03.514694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b976c5c00245fabdb8577b7c182e20', 'request_id': 'req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:03.529998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 839/1667] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:03 2022] PUT /identity/v3/projects/04b76c8ca55e44b3952d4d01a6916776/users/68c072ebb2c44d259e833f6cfa148807/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:03.531118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:03.543889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:03.549942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] The token cache is being invalidate because project c6b976c5c00245fabdb8577b7c182e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:03.550248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:03.552315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-17a2ddac-deaa-4e2f-ad34-908bd77a7d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:03.554355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 829/1668] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:03 2022] DELETE /identity/v3/projects/c6b976c5c00245fabdb8577b7c182e20 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:03.556403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2660746-c0f4-4a2a-8bfd-07da357a5f3f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.556991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2660746-c0f4-4a2a-8bfd-07da357a5f3f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.557316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2660746-c0f4-4a2a-8bfd-07da357a5f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.641596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f2660746-c0f4-4a2a-8bfd-07da357a5f3f None None] MFA Rules not processed for user `68c072ebb2c44d259e833f6cfa148807`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:03.645829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f2660746-c0f4-4a2a-8bfd-07da357a5f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:03.709659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4698483-feda-4113-b094-cc041472fef5 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:03.716757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4257818-6b36-4831-aade-d210ecbaf7e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:03.717405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4257818-6b36-4831-aade-d210ecbaf7e2 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.717722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4257818-6b36-4831-aade-d210ecbaf7e2 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.718000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4257818-6b36-4831-aade-d210ecbaf7e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.719879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d4257818-6b36-4831-aade-d210ecbaf7e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:03.736657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4257818-6b36-4831-aade-d210ecbaf7e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:03.738688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4257818-6b36-4831-aade-d210ecbaf7e2 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:03.749929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 830/1669] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:03.751272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb882b0-21b5-41d8-938f-08c94c48e20f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.753743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb882b0-21b5-41d8-938f-08c94c48e20f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.754218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb882b0-21b5-41d8-938f-08c94c48e20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.760320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 840/1670] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:03.762008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadb32b-1002-4b5c-b08b-aa6da078e10c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.762307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadb32b-1002-4b5c-b08b-aa6da078e10c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.762589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadb32b-1002-4b5c-b08b-aa6da078e10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.783513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2fb882b0-21b5-41d8-938f-08c94c48e20f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:03.789942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2fb882b0-21b5-41d8-938f-08c94c48e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:03.797475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-beadb32b-1002-4b5c-b08b-aa6da078e10c None None] MFA Rules not processed for user `a0e295fda2af4157b196c6130aa2d6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:03.798789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-beadb32b-1002-4b5c-b08b-aa6da078e10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:03.871345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 831/1671] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:03.872849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb882b0-21b5-41d8-938f-08c94c48e20f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:03.874465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2fb882b0-21b5-41d8-938f-08c94c48e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:03.874653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 841/1672] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:03.895553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216a9a-4d8f-4151-8fa6-f761296b5f4f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.896031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216a9a-4d8f-4151-8fa6-f761296b5f4f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.896454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216a9a-4d8f-4151-8fa6-f761296b5f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.910554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb882b0-21b5-41d8-938f-08c94c48e20f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:03.920485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912fd0c-977f-4031-ad44-1426465cb847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:03.921404 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912fd0c-977f-4031-ad44-1426465cb847 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:03.921836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912fd0c-977f-4031-ad44-1426465cb847 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:03.922208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912fd0c-977f-4031-ad44-1426465cb847 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:03.922959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912fd0c-977f-4031-ad44-1426465cb847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:03.931155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912fd0c-977f-4031-ad44-1426465cb847 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:03.946181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-75216a9a-4d8f-4151-8fa6-f761296b5f4f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:03.947416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-75216a9a-4d8f-4151-8fa6-f761296b5f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:03.965066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 842/1673] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:03.969769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75216a9a-4d8f-4151-8fa6-f761296b5f4f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:03.991034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75216a9a-4d8f-4151-8fa6-f761296b5f4f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.008627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.009340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.013358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.013546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.015461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:04.015461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.018842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.068242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 843/1674] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:04.072986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.089389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594a402-2205-4719-9fc1-7cbc222964c5 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.106044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcce0c-c40e-4253-9c7b-1907792fe023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.106681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcce0c-c40e-4253-9c7b-1907792fe023 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.106966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcce0c-c40e-4253-9c7b-1907792fe023 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.107418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcce0c-c40e-4253-9c7b-1907792fe023 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.110068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcce0c-c40e-4253-9c7b-1907792fe023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.114464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcce0c-c40e-4253-9c7b-1907792fe023 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.118359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 832/1675] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:03 2022] POST /identity/v3/projects => generated 387 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:04.119826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912fd0c-977f-4031-ad44-1426465cb847 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.137947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:04.198390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 844/1676] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:04 2022] POST /identity/v3/projects => generated 403 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:04.202552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcce0c-c40e-4253-9c7b-1907792fe023 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.224701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:04.286605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912fd0c-977f-4031-ad44-1426465cb847 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.380371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcce0c-c40e-4253-9c7b-1907792fe023 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.393781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d04a48-8996-46c8-a3c7-1a6e8670320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.394577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d04a48-8996-46c8-a3c7-1a6e8670320f admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.394857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d04a48-8996-46c8-a3c7-1a6e8670320f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.395137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d04a48-8996-46c8-a3c7-1a6e8670320f admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.397941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d04a48-8996-46c8-a3c7-1a6e8670320f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.404714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d04a48-8996-46c8-a3c7-1a6e8670320f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.438628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642294ba-ce6c-4a18-9cf6-411170e41fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.439329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642294ba-ce6c-4a18-9cf6-411170e41fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.439811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642294ba-ce6c-4a18-9cf6-411170e41fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.440128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642294ba-ce6c-4a18-9cf6-411170e41fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.441328 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642294ba-ce6c-4a18-9cf6-411170e41fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.442886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642294ba-ce6c-4a18-9cf6-411170e41fc5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.453530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 845/1677] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:04 2022] POST /identity/v3/users => generated 302 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:04.457196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d04a48-8996-46c8-a3c7-1a6e8670320f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.482301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d04a48-8996-46c8-a3c7-1a6e8670320f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.506850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cddc4-84d9-421e-9fc3-e747dfdd89f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.514822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 833/1678] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:04 2022] POST /identity/v3/users => generated 300 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:04.517542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cddc4-84d9-421e-9fc3-e747dfdd89f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.517765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cddc4-84d9-421e-9fc3-e747dfdd89f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.518028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cddc4-84d9-421e-9fc3-e747dfdd89f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.518725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cddc4-84d9-421e-9fc3-e747dfdd89f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.519268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642294ba-ce6c-4a18-9cf6-411170e41fc5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.520384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cddc4-84d9-421e-9fc3-e747dfdd89f5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.542140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642294ba-ce6c-4a18-9cf6-411170e41fc5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.552784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510226c-45ba-4ee2-aa0a-c502b7b8fbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.553801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510226c-45ba-4ee2-aa0a-c502b7b8fbde admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.554293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510226c-45ba-4ee2-aa0a-c502b7b8fbde admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.555303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510226c-45ba-4ee2-aa0a-c502b7b8fbde admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.556145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510226c-45ba-4ee2-aa0a-c502b7b8fbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.564056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 846/1679] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:04.566768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510226c-45ba-4ee2-aa0a-c502b7b8fbde admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.568333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cddc4-84d9-421e-9fc3-e747dfdd89f5 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.590471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 834/1680] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:04.590671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cddc4-84d9-421e-9fc3-e747dfdd89f5 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.594386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510226c-45ba-4ee2-aa0a-c502b7b8fbde admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.612689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510226c-45ba-4ee2-aa0a-c502b7b8fbde admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.617967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525176-9c3e-45b1-9ca4-013e1e4839b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.618730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525176-9c3e-45b1-9ca4-013e1e4839b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.620895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525176-9c3e-45b1-9ca4-013e1e4839b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.621731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be525176-9c3e-45b1-9ca4-013e1e4839b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.623162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be525176-9c3e-45b1-9ca4-013e1e4839b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.630194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be525176-9c3e-45b1-9ca4-013e1e4839b5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.634106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec796a-b6c6-4750-9a18-96181d8af7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.634934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec796a-b6c6-4750-9a18-96181d8af7bd admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.637313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec796a-b6c6-4750-9a18-96181d8af7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.637890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec796a-b6c6-4750-9a18-96181d8af7bd admin admin] PATH_INFO: `/v3/projects/a165c1081bc145a3a2b47d75831873fa/users/9947bd5bbebb435bb02bea04edf8e9aa/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.670136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 847/1681] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:04.679874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525176-9c3e-45b1-9ca4-013e1e4839b5 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.698176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec796a-b6c6-4750-9a18-96181d8af7bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a165c1081bc145a3a2b47d75831873fa, user_id=9947bd5bbebb435bb02bea04edf8e9aa, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.707828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec796a-b6c6-4750-9a18-96181d8af7bd admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.709965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be525176-9c3e-45b1-9ca4-013e1e4839b5 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:04.728432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b2d9f-d905-42ac-a5aa-c21cb265af32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:04.735799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b2d9f-d905-42ac-a5aa-c21cb265af32 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.737329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b2d9f-d905-42ac-a5aa-c21cb265af32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.738909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b2d9f-d905-42ac-a5aa-c21cb265af32 admin admin] PATH_INFO: `/v3/projects/f5a7bc692905432eba5daa939ada5322/users/758d987eba184dc88ddaba8cb8de45b3/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:04.747080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 835/1682] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:04 2022] PUT /identity/v3/projects/a165c1081bc145a3a2b47d75831873fa/users/9947bd5bbebb435bb02bea04edf8e9aa/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:04.769874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec796a-b6c6-4750-9a18-96181d8af7bd admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:04.834999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b2d9f-d905-42ac-a5aa-c21cb265af32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a7bc692905432eba5daa939ada5322, user_id=758d987eba184dc88ddaba8cb8de45b3, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:04.845303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b2d9f-d905-42ac-a5aa-c21cb265af32 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:04.898849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 848/1683] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:04 2022] PUT /identity/v3/projects/f5a7bc692905432eba5daa939ada5322/users/758d987eba184dc88ddaba8cb8de45b3/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:04.916379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e02d0-e087-472e-8909-6d6e27a45e64 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:04.916681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e02d0-e087-472e-8909-6d6e27a45e64 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:04.916994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e02d0-e087-472e-8909-6d6e27a45e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.022912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-397e02d0-e087-472e-8909-6d6e27a45e64 None None] MFA Rules not processed for user `758d987eba184dc88ddaba8cb8de45b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:05.024195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-397e02d0-e087-472e-8909-6d6e27a45e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:05.053112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec796a-b6c6-4750-9a18-96181d8af7bd admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:05.088886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:05.213043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcdcbc-2438-4805-94dd-1265e3c674cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:05.213782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcdcbc-2438-4805-94dd-1265e3c674cf admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:05.214180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcdcbc-2438-4805-94dd-1265e3c674cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:05.214705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcdcbc-2438-4805-94dd-1265e3c674cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.215527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcdcbc-2438-4805-94dd-1265e3c674cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:05.217636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcdcbc-2438-4805-94dd-1265e3c674cf admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:05.245284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 836/1684] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:05.250401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcdcbc-2438-4805-94dd-1265e3c674cf admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:05.254638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 849/1685] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:05.258700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e02d0-e087-472e-8909-6d6e27a45e64 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:05.268320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcdcbc-2438-4805-94dd-1265e3c674cf admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:05.278627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e02d0-e087-472e-8909-6d6e27a45e64 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:05.279418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5f1ac-0309-4f5a-a9d4-026eeda8c5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:05.280588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5f1ac-0309-4f5a-a9d4-026eeda8c5c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:05.281065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5f1ac-0309-4f5a-a9d4-026eeda8c5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:05.281700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5f1ac-0309-4f5a-a9d4-026eeda8c5c3 admin admin] PATH_INFO: `/v3/projects/a165c1081bc145a3a2b47d75831873fa/users/9947bd5bbebb435bb02bea04edf8e9aa/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.286015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5f1ac-0309-4f5a-a9d4-026eeda8c5c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a165c1081bc145a3a2b47d75831873fa, user_id=9947bd5bbebb435bb02bea04edf8e9aa, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:05.288495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e293dd-d824-46e0-8e90-e43478686e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:05.290437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5f1ac-0309-4f5a-a9d4-026eeda8c5c3 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:05.297276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e293dd-d824-46e0-8e90-e43478686e86 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:05.297568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e293dd-d824-46e0-8e90-e43478686e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:05.297918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e293dd-d824-46e0-8e90-e43478686e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.301801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e293dd-d824-46e0-8e90-e43478686e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:05.306125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e293dd-d824-46e0-8e90-e43478686e86 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:05.312459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 837/1686] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:05 2022] PUT /identity/v3/projects/a165c1081bc145a3a2b47d75831873fa/users/9947bd5bbebb435bb02bea04edf8e9aa/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:05.327698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7d06-f8a3-49f6-bb60-d7cdfa844652 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:05.327992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7d06-f8a3-49f6-bb60-d7cdfa844652 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:05.328269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b7d06-f8a3-49f6-bb60-d7cdfa844652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.376232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 850/1687] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:05 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:05.380382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e293dd-d824-46e0-8e90-e43478686e86 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:05.388292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-cf6b7d06-f8a3-49f6-bb60-d7cdfa844652 None None] MFA Rules not processed for user `9947bd5bbebb435bb02bea04edf8e9aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:05.389548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cf6b7d06-f8a3-49f6-bb60-d7cdfa844652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:05.434836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:05.498004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 838/1688] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:05.505904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b7d06-f8a3-49f6-bb60-d7cdfa844652 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:05.508515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cf6b7d06-f8a3-49f6-bb60-d7cdfa844652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:05.526048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e293dd-d824-46e0-8e90-e43478686e86 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:05.533049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6f76-9fa9-4d2e-8b16-97b7f0a22e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:05.534684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6f76-9fa9-4d2e-8b16-97b7f0a22e5a admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:05.534684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6f76-9fa9-4d2e-8b16-97b7f0a22e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:05.534885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a6f76-9fa9-4d2e-8b16-97b7f0a22e5a admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.535569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a6f76-9fa9-4d2e-8b16-97b7f0a22e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:05.538711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a6f76-9fa9-4d2e-8b16-97b7f0a22e5a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:05.576906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 851/1689] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:05 2022] POST /identity/v3/users => generated 309 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:05.582182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6f76-9fa9-4d2e-8b16-97b7f0a22e5a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:05.613108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a6f76-9fa9-4d2e-8b16-97b7f0a22e5a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:05.633307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f3a36-9056-4203-9b83-c51481ea74a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:05.633973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f3a36-9056-4203-9b83-c51481ea74a3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:05.634258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f3a36-9056-4203-9b83-c51481ea74a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:05.634537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f3a36-9056-4203-9b83-c51481ea74a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.638538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f3a36-9056-4203-9b83-c51481ea74a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:05.647282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f3a36-9056-4203-9b83-c51481ea74a3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:05.686894 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 852/1690] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:05.694316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f3a36-9056-4203-9b83-c51481ea74a3 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:05.696428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b7d06-f8a3-49f6-bb60-d7cdfa844652 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:05.712706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f5756-b08b-4e3f-a1dc-2831ecdbf9b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:05.713501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5756-b08b-4e3f-a1dc-2831ecdbf9b8 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:05.713906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5756-b08b-4e3f-a1dc-2831ecdbf9b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:05.714277 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5756-b08b-4e3f-a1dc-2831ecdbf9b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.724226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f3a36-9056-4203-9b83-c51481ea74a3 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:05.740777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00196f47-f6b3-4b88-8d42-c49bd9cf04d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:05.741706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00196f47-f6b3-4b88-8d42-c49bd9cf04d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:05.743305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00196f47-f6b3-4b88-8d42-c49bd9cf04d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:05.743757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00196f47-f6b3-4b88-8d42-c49bd9cf04d8 admin admin] PATH_INFO: `/v3/projects/a55616e17676497e91c9e8e1cd77eb94/users/efc69dcf954446e699bc45c396f4d59e/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:05.765651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f5756-b08b-4e3f-a1dc-2831ecdbf9b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:05.767445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f5756-b08b-4e3f-a1dc-2831ecdbf9b8 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:05.788384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 839/1691] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:05.843259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00196f47-f6b3-4b88-8d42-c49bd9cf04d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a55616e17676497e91c9e8e1cd77eb94, user_id=efc69dcf954446e699bc45c396f4d59e, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:05.849704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00196f47-f6b3-4b88-8d42-c49bd9cf04d8 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:05.886562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f5756-b08b-4e3f-a1dc-2831ecdbf9b8 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:05.902019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 853/1692] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:05 2022] PUT /identity/v3/projects/a55616e17676497e91c9e8e1cd77eb94/users/efc69dcf954446e699bc45c396f4d59e/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:05.906076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00196f47-f6b3-4b88-8d42-c49bd9cf04d8 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:05.917206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f5756-b08b-4e3f-a1dc-2831ecdbf9b8 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:05.992633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00196f47-f6b3-4b88-8d42-c49bd9cf04d8 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:06.002439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a216b-2e48-4f11-ae16-8c8e1dc902df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:06.003650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:06.004122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a216b-2e48-4f11-ae16-8c8e1dc902df admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:06.005194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a216b-2e48-4f11-ae16-8c8e1dc902df admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:06.006287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a216b-2e48-4f11-ae16-8c8e1dc902df admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:06.007600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a216b-2e48-4f11-ae16-8c8e1dc902df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:06.010799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a216b-2e48-4f11-ae16-8c8e1dc902df admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:06.050378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 854/1693] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:06.052311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a216b-2e48-4f11-ae16-8c8e1dc902df admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:06.105598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a216b-2e48-4f11-ae16-8c8e1dc902df admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:06.128957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70cfd37-e816-48e9-bdf0-e8961e0d5833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:06.132122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cfd37-e816-48e9-bdf0-e8961e0d5833 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:06.132412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cfd37-e816-48e9-bdf0-e8961e0d5833 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:06.132729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70cfd37-e816-48e9-bdf0-e8961e0d5833 admin admin] PATH_INFO: `/v3/projects/a55616e17676497e91c9e8e1cd77eb94/users/efc69dcf954446e699bc45c396f4d59e/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:06.140571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cfd37-e816-48e9-bdf0-e8961e0d5833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a55616e17676497e91c9e8e1cd77eb94, user_id=efc69dcf954446e699bc45c396f4d59e, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:06.152576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70cfd37-e816-48e9-bdf0-e8961e0d5833 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:06.208516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:06.209531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:06.209846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:06.210122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] PATH_INFO: `/v3/users/68c072ebb2c44d259e833f6cfa148807` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:06.214873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c072ebb2c44d259e833f6cfa148807)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:06.227877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 855/1694] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:06 2022] PUT /identity/v3/projects/a55616e17676497e91c9e8e1cd77eb94/users/efc69dcf954446e699bc45c396f4d59e/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:06.228098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:06.231181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173457f0-425d-43b4-89e2-0843fd7a33f2 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:06.231477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173457f0-425d-43b4-89e2-0843fd7a33f2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:06.231780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173457f0-425d-43b4-89e2-0843fd7a33f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:06.354577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-173457f0-425d-43b4-89e2-0843fd7a33f2 None None] MFA Rules not processed for user `efc69dcf954446e699bc45c396f4d59e`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:06.358515 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-173457f0-425d-43b4-89e2-0843fd7a33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:06.454884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c072ebb2c44d259e833f6cfa148807'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:06.487956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c072ebb2c44d259e833f6cfa148807'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:06.489224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c072ebb2c44d259e833f6cfa148807'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:06.533572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c072ebb2c44d259e833f6cfa148807'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:06.572019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-89a1c086-af52-4050-a9ae-6abeca82c37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c072ebb2c44d259e833f6cfa148807'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:06.591580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 840/1695] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:05 2022] DELETE /identity/v3/users/68c072ebb2c44d259e833f6cfa148807 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:06.593174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b8fa3-d98a-4744-85b7-062f81bac94e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:06.593467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b8fa3-d98a-4744-85b7-062f81bac94e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:06.605555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b8fa3-d98a-4744-85b7-062f81bac94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:06.636940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:06.674402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-874b8fa3-d98a-4744-85b7-062f81bac94e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:06.675421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-874b8fa3-d98a-4744-85b7-062f81bac94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:06.740025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 856/1696] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:06 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:06.760780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173457f0-425d-43b4-89e2-0843fd7a33f2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:06.763456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-173457f0-425d-43b4-89e2-0843fd7a33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:06.930790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 841/1697] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:06.986666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173457f0-425d-43b4-89e2-0843fd7a33f2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:07.223119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5ebb2-b6e4-4c0e-b58b-2cee708ea57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:07.223708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ebb2-b6e4-4c0e-b58b-2cee708ea57a service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:07.224348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ebb2-b6e4-4c0e-b58b-2cee708ea57a service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:07.224348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5ebb2-b6e4-4c0e-b58b-2cee708ea57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:07.379395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5ebb2-b6e4-4c0e-b58b-2cee708ea57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:07.379843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b8fa3-d98a-4744-85b7-062f81bac94e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:07.381051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5ebb2-b6e4-4c0e-b58b-2cee708ea57a service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:07.383268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-874b8fa3-d98a-4744-85b7-062f81bac94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:07.407249 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 857/1698] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:06 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:07.462430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c5284-0bfe-4b1c-b4b8-63c9bb7c0288 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:07.462971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c5284-0bfe-4b1c-b4b8-63c9bb7c0288 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:07.464862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c5284-0bfe-4b1c-b4b8-63c9bb7c0288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:07.499698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:07.523856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b8fa3-d98a-4744-85b7-062f81bac94e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:07.550310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:07.555164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:07.557535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:07.558417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:07.584198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:07.588033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bb7c5284-0bfe-4b1c-b4b8-63c9bb7c0288 None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:07.589425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bb7c5284-0bfe-4b1c-b4b8-63c9bb7c0288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:07.592177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:07.605206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 842/1699] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:07.624640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:07.745393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:07.747408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:07.765559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 858/1700] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:49:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:07.799058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340e312-c6f0-4e3b-b52b-0201031305c9 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:07.834024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:07.834698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:07.834981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:07.835256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] PATH_INFO: `/v3/users/758d987eba184dc88ddaba8cb8de45b3` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:07.838800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=758d987eba184dc88ddaba8cb8de45b3)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:07.840947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:07.986249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758d987eba184dc88ddaba8cb8de45b3'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:08.012914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758d987eba184dc88ddaba8cb8de45b3'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:08.015012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758d987eba184dc88ddaba8cb8de45b3'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:08.025691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c5284-0bfe-4b1c-b4b8-63c9bb7c0288 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:08.030180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bb7c5284-0bfe-4b1c-b4b8-63c9bb7c0288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:08.062369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758d987eba184dc88ddaba8cb8de45b3'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:08.081081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cd8612c0-5ab1-4fac-a7ae-e764251ccdcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758d987eba184dc88ddaba8cb8de45b3'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:08.099030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 843/1701] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:07 2022] DELETE /identity/v3/users/758d987eba184dc88ddaba8cb8de45b3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:08.119949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8ce9-35ec-48b2-87cf-1e97787eedf9 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:08.120713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8ce9-35ec-48b2-87cf-1e97787eedf9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:08.122017 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf8ce9-35ec-48b2-87cf-1e97787eedf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:08.181222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-dcdf8ce9-35ec-48b2-87cf-1e97787eedf9 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:08.182771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dcdf8ce9-35ec-48b2-87cf-1e97787eedf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:08.270656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c5284-0bfe-4b1c-b4b8-63c9bb7c0288 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:08.364096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 844/1702] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:08.408172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf8ce9-35ec-48b2-87cf-1e97787eedf9 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:08.411325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dcdf8ce9-35ec-48b2-87cf-1e97787eedf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:08.443684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:08.444537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:08.445518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:08.445935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:08.450679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:08.636013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:08.637700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf8ce9-35ec-48b2-87cf-1e97787eedf9 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:08.659407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba963223-b3c6-4ac0-8a85-b12a5d9b7cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:08.660497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba963223-b3c6-4ac0-8a85-b12a5d9b7cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:08.660999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba963223-b3c6-4ac0-8a85-b12a5d9b7cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:08.662948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba963223-b3c6-4ac0-8a85-b12a5d9b7cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:08.693824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:08.696021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:08.713885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 859/1703] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 19:49:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:49:08.721642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba963223-b3c6-4ac0-8a85-b12a5d9b7cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:08.723172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba963223-b3c6-4ac0-8a85-b12a5d9b7cb1 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:08.738678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:08.763762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 845/1704] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:11.612795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:11.614349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:11.653810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064665f4-86d0-4ec1-ad91-30030afaa23a service swift] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:11.674825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79324b2c-29ab-4fec-8261-6a1448762ded None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:11.675349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79324b2c-29ab-4fec-8261-6a1448762ded None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:11.675842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79324b2c-29ab-4fec-8261-6a1448762ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:11.680840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:11.682123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:11.683013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:11.684296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] PATH_INFO: `/v3/projects/04b76c8ca55e44b3952d4d01a6916776` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:11.699032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b76c8ca55e44b3952d4d01a6916776)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:11.704113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:11.775362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-79324b2c-29ab-4fec-8261-6a1448762ded None None] MFA Rules not processed for user `0cf74b3dd4c24c1aa3c0ac659b37c0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:11.777633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-79324b2c-29ab-4fec-8261-6a1448762ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:11.880096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b76c8ca55e44b3952d4d01a6916776', 'request_id': 'req-e3bfa606-61a6-495a-9680-c581799fc1e5'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:11.920169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b76c8ca55e44b3952d4d01a6916776', 'request_id': 'req-e3bfa606-61a6-495a-9680-c581799fc1e5'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:11.940721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b76c8ca55e44b3952d4d01a6916776', 'request_id': 'req-e3bfa606-61a6-495a-9680-c581799fc1e5'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:11.978389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] The token cache is being invalidate because project 04b76c8ca55e44b3952d4d01a6916776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:11.979407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:11.982336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:11.992543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 860/1705] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:11 2022] DELETE /identity/v3/projects/04b76c8ca55e44b3952d4d01a6916776 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:11.993584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:12.000170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:12.057039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 846/1706] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:12.068790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaf2c4-4881-4708-b48f-122fbcae87f0 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:12.071086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaf2c4-4881-4708-b48f-122fbcae87f0 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:12.071488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baaf2c4-4881-4708-b48f-122fbcae87f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:12.158670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-6baaf2c4-4881-4708-b48f-122fbcae87f0 None None] MFA Rules not processed for user `0cf74b3dd4c24c1aa3c0ac659b37c0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:12.161218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6baaf2c4-4881-4708-b48f-122fbcae87f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:12.292132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bfa606-61a6-495a-9680-c581799fc1e5 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:12.307182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:12.308034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:12.308435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:12.308817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] PATH_INFO: `/v3/users/9947bd5bbebb435bb02bea04edf8e9aa` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:12.310472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9947bd5bbebb435bb02bea04edf8e9aa)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:12.316603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:12.401244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 847/1707] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:12.415757 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaf2c4-4881-4708-b48f-122fbcae87f0 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:12.420962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6baaf2c4-4881-4708-b48f-122fbcae87f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:12.455387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9947bd5bbebb435bb02bea04edf8e9aa'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:12.486063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9947bd5bbebb435bb02bea04edf8e9aa'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:12.486849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9947bd5bbebb435bb02bea04edf8e9aa'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:12.509663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:12.517491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9947bd5bbebb435bb02bea04edf8e9aa'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:12.551364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9947bd5bbebb435bb02bea04edf8e9aa'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:12.568614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 861/1708] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:11 2022] DELETE /identity/v3/users/9947bd5bbebb435bb02bea04edf8e9aa => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:12.835140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baaf2c4-4881-4708-b48f-122fbcae87f0 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:12.998884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:13.067665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:13.069473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:13.221051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:13.221767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:13.222052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:13.222331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:13.234306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:13.346767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c91e-2daa-402a-8ab7-3474d17e918c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:13.372505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:13.372979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:13.373586 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:13.374297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] PATH_INFO: `/v3/projects/f5a7bc692905432eba5daa939ada5322` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:13.382725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a7bc692905432eba5daa939ada5322)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:13.385814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:13.490546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:13.493564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:13.523263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a7bc692905432eba5daa939ada5322', 'request_id': 'req-8886c986-f1c4-4640-87e9-efb135cd16ab'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:13.530816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 848/1709] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 1116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:13.571546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a7bc692905432eba5daa939ada5322', 'request_id': 'req-8886c986-f1c4-4640-87e9-efb135cd16ab'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:13.590517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a7bc692905432eba5daa939ada5322', 'request_id': 'req-8886c986-f1c4-4640-87e9-efb135cd16ab'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:13.623067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] The token cache is being invalidate because project f5a7bc692905432eba5daa939ada5322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:13.623537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:13.625117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8886c986-f1c4-4640-87e9-efb135cd16ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:13.635579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 862/1710] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:13 2022] DELETE /identity/v3/projects/f5a7bc692905432eba5daa939ada5322 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:13.636783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:13.662489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49447299-6d55-4ad7-953d-efeb20f8bc11 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:13.839725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:13.840490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:13.841287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:13.841777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] PATH_INFO: `/v3/users/efc69dcf954446e699bc45c396f4d59e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:13.844472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] RBAC: Authorizing `identity:delete_user(user_id=efc69dcf954446e699bc45c396f4d59e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:13.846482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:13.965057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efc69dcf954446e699bc45c396f4d59e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:13.984152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efc69dcf954446e699bc45c396f4d59e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:13.989637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efc69dcf954446e699bc45c396f4d59e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:14.041264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579b263-ec78-4e6b-98e2-3b53368ecab9 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:14.041829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579b263-ec78-4e6b-98e2-3b53368ecab9 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:14.042252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579b263-ec78-4e6b-98e2-3b53368ecab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:14.052715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efc69dcf954446e699bc45c396f4d59e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:14.054390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:14.075338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efc69dcf954446e699bc45c396f4d59e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:14.085943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 849/1711] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:13 2022] DELETE /identity/v3/users/efc69dcf954446e699bc45c396f4d59e => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:14.101627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-0579b263-ec78-4e6b-98e2-3b53368ecab9 None None] MFA Rules not processed for user `9f77e57b2d96461c88e2fc33db22e22d`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:14.103010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0579b263-ec78-4e6b-98e2-3b53368ecab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:14.269527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 863/1712] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Sep 13 19:49:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:14.299678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:14.301687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:14.332295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decee6b1-c2b3-4e1e-9584-e56b90ba296c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:14.343395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de23d1d-1c74-4b49-a35d-16260fb5d9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:14.343943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de23d1d-1c74-4b49-a35d-16260fb5d9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:14.344224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de23d1d-1c74-4b49-a35d-16260fb5d9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:14.344503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de23d1d-1c74-4b49-a35d-16260fb5d9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:14.358205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de23d1d-1c74-4b49-a35d-16260fb5d9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:14.359814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de23d1d-1c74-4b49-a35d-16260fb5d9f3 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:14.378910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 850/1713] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:15.133687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad5e47-dac8-40c8-a824-0942c92488c6 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:15.134769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad5e47-dac8-40c8-a824-0942c92488c6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:15.135206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad5e47-dac8-40c8-a824-0942c92488c6 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:15.137304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 864/1714] 10.222.0.23 () {54 vars in 912 bytes} [Tue Sep 13 19:49:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:15.143839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab092-4f23-41e0-9ea9-897f2e661305 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:15.144798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab092-4f23-41e0-9ea9-897f2e661305 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:15.152304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfab092-4f23-41e0-9ea9-897f2e661305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:15.199324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-bcfab092-4f23-41e0-9ea9-897f2e661305 None None] MFA Rules not processed for user `9f77e57b2d96461c88e2fc33db22e22d`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:15.203028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bcfab092-4f23-41e0-9ea9-897f2e661305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:15.239528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 851/1715] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:49:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:15.344183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad5e47-dac8-40c8-a824-0942c92488c6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:15.347839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-52ad5e47-dac8-40c8-a824-0942c92488c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:15.387961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad5e47-dac8-40c8-a824-0942c92488c6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:15.431499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce925313-3fe4-4973-b41f-0247bd9c95f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:15.432419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce925313-3fe4-4973-b41f-0247bd9c95f6 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:15.432881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce925313-3fe4-4973-b41f-0247bd9c95f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:15.433311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce925313-3fe4-4973-b41f-0247bd9c95f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:15.461709 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce925313-3fe4-4973-b41f-0247bd9c95f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:15.463443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce925313-3fe4-4973-b41f-0247bd9c95f6 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:15.479918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 865/1716] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:16.144525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfab092-4f23-41e0-9ea9-897f2e661305 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:16.146218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bcfab092-4f23-41e0-9ea9-897f2e661305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:16.384365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfab092-4f23-41e0-9ea9-897f2e661305 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:16.422040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:16.424344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:16.427024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:16.429905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] PATH_INFO: `/v3/projects/a165c1081bc145a3a2b47d75831873fa` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:16.434626 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a165c1081bc145a3a2b47d75831873fa)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:16.440703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:16.719360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a165c1081bc145a3a2b47d75831873fa', 'request_id': 'req-32913ccb-ad6f-4170-bb0d-dc47da379e19'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:16.736238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a165c1081bc145a3a2b47d75831873fa', 'request_id': 'req-32913ccb-ad6f-4170-bb0d-dc47da379e19'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:16.782508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a165c1081bc145a3a2b47d75831873fa', 'request_id': 'req-32913ccb-ad6f-4170-bb0d-dc47da379e19'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:16.895132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] The token cache is being invalidate because project a165c1081bc145a3a2b47d75831873fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:16.896772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:16.907405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:16.926509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 852/1717] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:16 2022] DELETE /identity/v3/projects/a165c1081bc145a3a2b47d75831873fa => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:16.932685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a2e80-96b9-47aa-b51d-fc70cab90815 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:16.933629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a2e80-96b9-47aa-b51d-fc70cab90815 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:16.934077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a2e80-96b9-47aa-b51d-fc70cab90815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:16.999270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-cb8a2e80-96b9-47aa-b51d-fc70cab90815 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:17.000554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-cb8a2e80-96b9-47aa-b51d-fc70cab90815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:17.035537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:17.238753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:17.241009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 866/1718] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:17.245856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622dd3a-ebfc-4ec7-9ea8-87111c01f7ff None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:17.246385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622dd3a-ebfc-4ec7-9ea8-87111c01f7ff None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:17.247391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:17.247855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622dd3a-ebfc-4ec7-9ea8-87111c01f7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:17.299944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-6622dd3a-ebfc-4ec7-9ea8-87111c01f7ff None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:17.301781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6622dd3a-ebfc-4ec7-9ea8-87111c01f7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:17.335320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 867/1719] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:17.342037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622dd3a-ebfc-4ec7-9ea8-87111c01f7ff None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:17.357781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32913ccb-ad6f-4170-bb0d-dc47da379e19 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:17.390866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622dd3a-ebfc-4ec7-9ea8-87111c01f7ff None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:17.408620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:17.409300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:17.409628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:17.409951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] PATH_INFO: `/v3/projects/a55616e17676497e91c9e8e1cd77eb94` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:17.412921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:17.418789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:17.419898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:17.421508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a55616e17676497e91c9e8e1cd77eb94)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:17.421920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:17.423610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:17.425032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:17.425997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:17.437244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:17.498709 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 868/1720] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:17.501410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:17.547317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1a79d-a4fe-4a19-a2c3-31112821d9d6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:17.576321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a55616e17676497e91c9e8e1cd77eb94', 'request_id': 'req-d954f894-a5fa-4f14-a43e-d41b8aeec786'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:17.581408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:17.584257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a55616e17676497e91c9e8e1cd77eb94', 'request_id': 'req-d954f894-a5fa-4f14-a43e-d41b8aeec786'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:17.598557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a55616e17676497e91c9e8e1cd77eb94', 'request_id': 'req-d954f894-a5fa-4f14-a43e-d41b8aeec786'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:17.623441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] The token cache is being invalidate because project a55616e17676497e91c9e8e1cd77eb94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:17.623996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:17.624851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d954f894-a5fa-4f14-a43e-d41b8aeec786 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:17.627188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 853/1721] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:17 2022] DELETE /identity/v3/projects/a55616e17676497e91c9e8e1cd77eb94 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:17.633590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018a6f-cd21-42ea-9645-9467070748a4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:17.634149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018a6f-cd21-42ea-9645-9467070748a4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:17.634581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73018a6f-cd21-42ea-9645-9467070748a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:17.677781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-73018a6f-cd21-42ea-9645-9467070748a4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:17.680284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-73018a6f-cd21-42ea-9645-9467070748a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:17.717891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:17.718701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:17.719092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:17.719463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:17.720212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:17.728317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:17.795061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 869/1722] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:17 2022] POST /identity/v3/projects => generated 387 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:17.795061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:17.806032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:17.860457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 854/1723] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:17.870020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5958-82a9-4247-9a17-d13aca9cc3e4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:17.870365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5958-82a9-4247-9a17-d13aca9cc3e4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:17.870777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a5958-82a9-4247-9a17-d13aca9cc3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:17.937927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c29a5958-82a9-4247-9a17-d13aca9cc3e4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:17.939313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c29a5958-82a9-4247-9a17-d13aca9cc3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:18.029103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d59d834-fc41-48c3-8685-389eb64d0301 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:18.054869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8546-3988-4574-bd1c-85d5eb2a199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:18.055469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8546-3988-4574-bd1c-85d5eb2a199b admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:18.055745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8546-3988-4574-bd1c-85d5eb2a199b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:18.056088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8546-3988-4574-bd1c-85d5eb2a199b admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:18.056659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d8546-3988-4574-bd1c-85d5eb2a199b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:18.058660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d8546-3988-4574-bd1c-85d5eb2a199b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:18.133291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 870/1724] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:17 2022] POST /identity/v3/users => generated 294 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:18.137001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8546-3988-4574-bd1c-85d5eb2a199b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:18.146510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 855/1725] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:18.147477 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a5958-82a9-4247-9a17-d13aca9cc3e4 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:18.167462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8546-3988-4574-bd1c-85d5eb2a199b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:18.177697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a5958-82a9-4247-9a17-d13aca9cc3e4 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:18.183360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66134b8-2d2f-4f5e-9335-a1b3126d03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:18.184553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66134b8-2d2f-4f5e-9335-a1b3126d03c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:18.185491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66134b8-2d2f-4f5e-9335-a1b3126d03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:18.186111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66134b8-2d2f-4f5e-9335-a1b3126d03c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:18.205159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66134b8-2d2f-4f5e-9335-a1b3126d03c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:18.208443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:18.209687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66134b8-2d2f-4f5e-9335-a1b3126d03c1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:18.212287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:18.214507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:18.214641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:18.214998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:18.218069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:18.218864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:18.220555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:18.280671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 871/1726] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:18.296685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427df0f-e845-4d80-9b00-374a6b55f248 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:18.297063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427df0f-e845-4d80-9b00-374a6b55f248 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:18.297393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427df0f-e845-4d80-9b00-374a6b55f248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:18.318088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 856/1727] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:18.326254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:18.374430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f69478-dba9-460d-8178-3f3c758d5107 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:18.419017 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb32da-3626-43fb-805e-c7630d706b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:18.422706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb32da-3626-43fb-805e-c7630d706b54 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:18.426012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb32da-3626-43fb-805e-c7630d706b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:18.427049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb32da-3626-43fb-805e-c7630d706b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:18.431231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb32da-3626-43fb-805e-c7630d706b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:18.448989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb32da-3626-43fb-805e-c7630d706b54 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:18.475820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9427df0f-e845-4d80-9b00-374a6b55f248 None None] MFA Rules not processed for user `15069aa932aa473983c4102961b855d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:18.484889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9427df0f-e845-4d80-9b00-374a6b55f248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:18.499616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 857/1728] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:18 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:18.500690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb32da-3626-43fb-805e-c7630d706b54 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:18.518580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb32da-3626-43fb-805e-c7630d706b54 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:18.525671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153bb6e-94fe-418d-ac73-6fe50f4eb337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:18.526311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153bb6e-94fe-418d-ac73-6fe50f4eb337 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:18.526595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153bb6e-94fe-418d-ac73-6fe50f4eb337 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:18.527213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153bb6e-94fe-418d-ac73-6fe50f4eb337 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:18.528230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153bb6e-94fe-418d-ac73-6fe50f4eb337 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:18.530316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153bb6e-94fe-418d-ac73-6fe50f4eb337 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:18.640907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 858/1729] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:18 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:18.646554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153bb6e-94fe-418d-ac73-6fe50f4eb337 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:18.785763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:18.838381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 872/1730] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 557 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:18.839376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427df0f-e845-4d80-9b00-374a6b55f248 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:18.858319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153bb6e-94fe-418d-ac73-6fe50f4eb337 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:18.883440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818421e-55a8-4c56-8125-4f9aa7fe20d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:18.886802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818421e-55a8-4c56-8125-4f9aa7fe20d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:18.892309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818421e-55a8-4c56-8125-4f9aa7fe20d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:18.892541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9427df0f-e845-4d80-9b00-374a6b55f248 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:18.892661 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818421e-55a8-4c56-8125-4f9aa7fe20d8 admin admin] PATH_INFO: `/v3/projects/37fa5f5c8f9740689c1434c673920fdf/users/a09ffb27443141bb97270189876ded74/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:18.911505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:18.912154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:18.912434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:18.912731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:18.913968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:18.915965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:18.966850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818421e-55a8-4c56-8125-4f9aa7fe20d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fa5f5c8f9740689c1434c673920fdf, user_id=a09ffb27443141bb97270189876ded74, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:18.969592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818421e-55a8-4c56-8125-4f9aa7fe20d8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:18.998965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 873/1731] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:18 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:18.998965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:19.026769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:19.051804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 859/1732] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:18 2022] PUT /identity/v3/projects/37fa5f5c8f9740689c1434c673920fdf/users/a09ffb27443141bb97270189876ded74/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:19.052481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818421e-55a8-4c56-8125-4f9aa7fe20d8 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:19.361410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad091385-2587-45fd-8951-c4240abe3680 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:19.416186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818421e-55a8-4c56-8125-4f9aa7fe20d8 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:19.455568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051ca78-5dfa-47ba-a5ee-29a71e7ad9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:19.461253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051ca78-5dfa-47ba-a5ee-29a71e7ad9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:19.461867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051ca78-5dfa-47ba-a5ee-29a71e7ad9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:19.462364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051ca78-5dfa-47ba-a5ee-29a71e7ad9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:19.463669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051ca78-5dfa-47ba-a5ee-29a71e7ad9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:19.469089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051ca78-5dfa-47ba-a5ee-29a71e7ad9f1 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:19.528586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 860/1733] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:19.532966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e50320-7fe2-4b2e-8b7b-04765450a90f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:19.538149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e50320-7fe2-4b2e-8b7b-04765450a90f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:19.538916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e50320-7fe2-4b2e-8b7b-04765450a90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:19.610487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:19.638606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed71db5-2d31-4a39-86a7-75466ff9176f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:19.639201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71db5-2d31-4a39-86a7-75466ff9176f service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:19.639487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71db5-2d31-4a39-86a7-75466ff9176f service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:19.639768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed71db5-2d31-4a39-86a7-75466ff9176f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:19.687399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-16e50320-7fe2-4b2e-8b7b-04765450a90f None None] MFA Rules not processed for user `a09ffb27443141bb97270189876ded74`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:19.688758 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-16e50320-7fe2-4b2e-8b7b-04765450a90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:19.849936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed71db5-2d31-4a39-86a7-75466ff9176f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:19.854843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed71db5-2d31-4a39-86a7-75466ff9176f service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:19.880092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 861/1734] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:19.881037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e50320-7fe2-4b2e-8b7b-04765450a90f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:19.886704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 874/1735] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 889 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:19.888174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8ad7-db32-4e7c-8d10-0742026c3e47 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:19.888466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8ad7-db32-4e7c-8d10-0742026c3e47 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:19.888576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-16e50320-7fe2-4b2e-8b7b-04765450a90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:19.888749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b8ad7-db32-4e7c-8d10-0742026c3e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:19.923356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e50320-7fe2-4b2e-8b7b-04765450a90f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:19.929419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:19.930199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:19.930481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:19.930847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] PATH_INFO: `/v3/users/123b601ea1f24cec8604de94e2099ffe` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:19.932089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=123b601ea1f24cec8604de94e2099ffe)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:19.934244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:20.003632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-8d8b8ad7-db32-4e7c-8d10-0742026c3e47 None None] MFA Rules not processed for user `61b9ae7499154e5eab4f3c23d8bb6785`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:20.008570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8d8b8ad7-db32-4e7c-8d10-0742026c3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:20.107309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:20.138121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123b601ea1f24cec8604de94e2099ffe'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:20.153631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123b601ea1f24cec8604de94e2099ffe'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:20.154561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123b601ea1f24cec8604de94e2099ffe'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:20.201337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123b601ea1f24cec8604de94e2099ffe'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:20.235327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123b601ea1f24cec8604de94e2099ffe'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:20.276361 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 862/1737] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:19 2022] DELETE /identity/v3/users/123b601ea1f24cec8604de94e2099ffe => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:20.276574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 875/1737] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:20.277523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8ad7-db32-4e7c-8d10-0742026c3e47 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:20.277813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:20.279832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8d8b8ad7-db32-4e7c-8d10-0742026c3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:20.280680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:20.315827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b8ad7-db32-4e7c-8d10-0742026c3e47 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:20.454473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d608e-c3d3-48b4-943c-395251411c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:20.468197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d608e-c3d3-48b4-943c-395251411c61 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:20.469401 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d608e-c3d3-48b4-943c-395251411c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:20.470695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d608e-c3d3-48b4-943c-395251411c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:20.473876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d608e-c3d3-48b4-943c-395251411c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:20.480906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d608e-c3d3-48b4-943c-395251411c61 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:20.546018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 876/1738] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:20.552438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd0b9-a608-46b6-9556-11ff31649a45 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:20.553586 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd0b9-a608-46b6-9556-11ff31649a45 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:20.553889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cbd0b9-a608-46b6-9556-11ff31649a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:20.560926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc574e-36d1-4307-b796-89d63a37ad54 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:20.579718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3941c-cb2c-4618-abbe-69b6153ade82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:20.580469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3941c-cb2c-4618-abbe-69b6153ade82 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:20.580793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3941c-cb2c-4618-abbe-69b6153ade82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:20.581150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3941c-cb2c-4618-abbe-69b6153ade82 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:20.581887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3941c-cb2c-4618-abbe-69b6153ade82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:20.584019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3941c-cb2c-4618-abbe-69b6153ade82 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:20.642066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-98cbd0b9-a608-46b6-9556-11ff31649a45 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:20.651567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-98cbd0b9-a608-46b6-9556-11ff31649a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:20.655138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 863/1739] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:20 2022] POST /identity/v3/projects => generated 387 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:20.656212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3941c-cb2c-4618-abbe-69b6153ade82 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:20.814460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:20.856199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3941c-cb2c-4618-abbe-69b6153ade82 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:20.871335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:20.872042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:20.872314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:20.872588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] PATH_INFO: `/v3/projects/9eb68c7843e44d3a93efcbdf9b9091e8/users/f638a8db83b84cd580aa3f81327bc719/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:20.901765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 877/1740] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:20.908243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbd0b9-a608-46b6-9556-11ff31649a45 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:20.909874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb68c7843e44d3a93efcbdf9b9091e8, user_id=f638a8db83b84cd580aa3f81327bc719, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:20.912681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:20.934223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cbd0b9-a608-46b6-9556-11ff31649a45 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:20.951215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 864/1741] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:20 2022] PUT /identity/v3/projects/9eb68c7843e44d3a93efcbdf9b9091e8/users/f638a8db83b84cd580aa3f81327bc719/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:20.952262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:20.959248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:21.035559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:21.058849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7902f6d-1151-4c80-9aee-9ca9e1ca4403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:21.065024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7902f6d-1151-4c80-9aee-9ca9e1ca4403 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:21.066183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7902f6d-1151-4c80-9aee-9ca9e1ca4403 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:21.067872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7902f6d-1151-4c80-9aee-9ca9e1ca4403 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:21.070385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7902f6d-1151-4c80-9aee-9ca9e1ca4403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:21.076179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7902f6d-1151-4c80-9aee-9ca9e1ca4403 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:21.201951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 878/1742] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:20 2022] POST /identity/v3/users => generated 294 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:21.213233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25e6fd-1c3b-4b95-9da2-ee34d0661bdf None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:21.213861 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25e6fd-1c3b-4b95-9da2-ee34d0661bdf None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:21.214171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25e6fd-1c3b-4b95-9da2-ee34d0661bdf None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:21.228023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 879/1743] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:49:21 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:21.228023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9858dd-a5ea-41df-a98e-267131307144 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:21.228023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9858dd-a5ea-41df-a98e-267131307144 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:21.231908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9858dd-a5ea-41df-a98e-267131307144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:21.360368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df72322-a438-4ece-bc49-7bafa28dd8e9 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:21.387493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95067937-dc60-41ce-9127-7977374c5a21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:21.387761 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4a9858dd-a5ea-41df-a98e-267131307144 None None] MFA Rules not processed for user `f638a8db83b84cd580aa3f81327bc719`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:21.388753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067937-dc60-41ce-9127-7977374c5a21 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:21.389222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067937-dc60-41ce-9127-7977374c5a21 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:21.389906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95067937-dc60-41ce-9127-7977374c5a21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:21.394581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4a9858dd-a5ea-41df-a98e-267131307144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:21.610880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95067937-dc60-41ce-9127-7977374c5a21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:21.612958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95067937-dc60-41ce-9127-7977374c5a21 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:21.679143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 865/1744] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:21.680087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95067937-dc60-41ce-9127-7977374c5a21 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:21.707571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95067937-dc60-41ce-9127-7977374c5a21 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:21.714125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c4057-1261-4d66-8ac2-6060676fe962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:21.715229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c4057-1261-4d66-8ac2-6060676fe962 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:21.715662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c4057-1261-4d66-8ac2-6060676fe962 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:21.716182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c4057-1261-4d66-8ac2-6060676fe962 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:21.716972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 880/1745] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:21.718013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c4057-1261-4d66-8ac2-6060676fe962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:21.720321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c4057-1261-4d66-8ac2-6060676fe962 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:21.731458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9858dd-a5ea-41df-a98e-267131307144 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:21.734297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4a9858dd-a5ea-41df-a98e-267131307144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:21.754858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 866/1746] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:21.756747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dc98e-77e3-490d-ae5a-cbe94a716e6f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:21.757188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dc98e-77e3-490d-ae5a-cbe94a716e6f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:21.757625 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dc98e-77e3-490d-ae5a-cbe94a716e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:21.782710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-047dc98e-77e3-490d-ae5a-cbe94a716e6f None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:21.784145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-047dc98e-77e3-490d-ae5a-cbe94a716e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:21.795687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 867/1747] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:49:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:21.796857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dc98e-77e3-490d-ae5a-cbe94a716e6f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:21.798761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-047dc98e-77e3-490d-ae5a-cbe94a716e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:21.878512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:21.894238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9858dd-a5ea-41df-a98e-267131307144 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:21.898485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bae577-4ce6-48d0-b68a-eb5a3c2a5807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:21.899173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae577-4ce6-48d0-b68a-eb5a3c2a5807 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:21.899461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae577-4ce6-48d0-b68a-eb5a3c2a5807 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:21.900038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bae577-4ce6-48d0-b68a-eb5a3c2a5807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:21.914915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bae577-4ce6-48d0-b68a-eb5a3c2a5807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:21.916555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bae577-4ce6-48d0-b68a-eb5a3c2a5807 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:21.929148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 881/1748] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:21.930284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bae577-4ce6-48d0-b68a-eb5a3c2a5807 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:21.942900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bae577-4ce6-48d0-b68a-eb5a3c2a5807 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:21.949834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba33a-55d8-408f-88e9-9d3042cfdfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:21.950484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba33a-55d8-408f-88e9-9d3042cfdfcb admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:21.950765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba33a-55d8-408f-88e9-9d3042cfdfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:21.951060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ba33a-55d8-408f-88e9-9d3042cfdfcb admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:21.951763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ba33a-55d8-408f-88e9-9d3042cfdfcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:21.954103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ba33a-55d8-408f-88e9-9d3042cfdfcb admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:21.971109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 882/1749] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:21.975113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba33a-55d8-408f-88e9-9d3042cfdfcb admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:22.002381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dc98e-77e3-490d-ae5a-cbe94a716e6f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:22.003920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ba33a-55d8-408f-88e9-9d3042cfdfcb admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:22.010017 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bc33b-90ce-4aa4-8996-1d24ffba3b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:22.010663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc33b-90ce-4aa4-8996-1d24ffba3b57 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:22.010946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc33b-90ce-4aa4-8996-1d24ffba3b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:22.011222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bc33b-90ce-4aa4-8996-1d24ffba3b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:22.033666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bc33b-90ce-4aa4-8996-1d24ffba3b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:22.035415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bc33b-90ce-4aa4-8996-1d24ffba3b57 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:22.046876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d9829-5a8d-4bf2-b50d-e59e47977bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:22.048920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d9829-5a8d-4bf2-b50d-e59e47977bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:22.049306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d9829-5a8d-4bf2-b50d-e59e47977bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:22.050741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d9829-5a8d-4bf2-b50d-e59e47977bd7 admin admin] PATH_INFO: `/v3/projects/5fc49c89aa8040e1a353944918139d3f/users/53a0ee56cb51446ca836587e3c3e7be6/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:22.078961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 868/1750] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:22.089396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d9829-5a8d-4bf2-b50d-e59e47977bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc49c89aa8040e1a353944918139d3f, user_id=53a0ee56cb51446ca836587e3c3e7be6, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:22.093438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d9829-5a8d-4bf2-b50d-e59e47977bd7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:22.118489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 883/1751] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:21 2022] PUT /identity/v3/projects/5fc49c89aa8040e1a353944918139d3f/users/53a0ee56cb51446ca836587e3c3e7be6/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:22.129266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a018-e45f-46b1-a83e-650b0b1107b3 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:22.129595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a018-e45f-46b1-a83e-650b0b1107b3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:22.129901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a018-e45f-46b1-a83e-650b0b1107b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:22.161187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d9829-5a8d-4bf2-b50d-e59e47977bd7 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:22.230761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-a796a018-e45f-46b1-a83e-650b0b1107b3 None None] MFA Rules not processed for user `53a0ee56cb51446ca836587e3c3e7be6`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:22.232376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a796a018-e45f-46b1-a83e-650b0b1107b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:22.362028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:22.391151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d9829-5a8d-4bf2-b50d-e59e47977bd7 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:22.406021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26954a90-e690-4d17-8c2d-c7bbe32d1841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:22.408399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26954a90-e690-4d17-8c2d-c7bbe32d1841 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:22.409191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26954a90-e690-4d17-8c2d-c7bbe32d1841 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:22.410151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26954a90-e690-4d17-8c2d-c7bbe32d1841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:22.438567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26954a90-e690-4d17-8c2d-c7bbe32d1841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:22.440928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26954a90-e690-4d17-8c2d-c7bbe32d1841 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:22.447033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 869/1752] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:22.451249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796a018-e45f-46b1-a83e-650b0b1107b3 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:22.472212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 884/1753] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:22.551339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796a018-e45f-46b1-a83e-650b0b1107b3 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:22.557387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f2560-44e8-4230-b405-efdf65ec2ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:22.558422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2560-44e8-4230-b405-efdf65ec2ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:22.558797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2560-44e8-4230-b405-efdf65ec2ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:22.559147 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f2560-44e8-4230-b405-efdf65ec2ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:22.560260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f2560-44e8-4230-b405-efdf65ec2ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:22.562941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f2560-44e8-4230-b405-efdf65ec2ce0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:22.662264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 870/1754] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:22 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:22.666773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26954a90-e690-4d17-8c2d-c7bbe32d1841 service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:22.691417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26954a90-e690-4d17-8c2d-c7bbe32d1841 service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:22.901968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fe121-8e4b-4952-8575-1405a7f50f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:22.902713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fe121-8e4b-4952-8575-1405a7f50f48 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:22.902875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fe121-8e4b-4952-8575-1405a7f50f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:22.903381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fe121-8e4b-4952-8575-1405a7f50f48 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:22.904077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fe121-8e4b-4952-8575-1405a7f50f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:22.909164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fe121-8e4b-4952-8575-1405a7f50f48 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:22.959224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 885/1755] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:22 2022] POST /identity/v3/users => generated 300 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:22.962784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f2560-44e8-4230-b405-efdf65ec2ce0 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:23.000042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f2560-44e8-4230-b405-efdf65ec2ce0 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:23.035580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241e3f-42bc-4934-a84f-22ad18d7b85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:23.040320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241e3f-42bc-4934-a84f-22ad18d7b85c admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:23.043248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241e3f-42bc-4934-a84f-22ad18d7b85c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:23.043672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e241e3f-42bc-4934-a84f-22ad18d7b85c admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:23.044512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e241e3f-42bc-4934-a84f-22ad18d7b85c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:23.049206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e241e3f-42bc-4934-a84f-22ad18d7b85c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:23.109465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 871/1756] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:23.114436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fe121-8e4b-4952-8575-1405a7f50f48 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:23.129309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fe121-8e4b-4952-8575-1405a7f50f48 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:23.156405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29680e-cf1f-42ff-a2e0-19aa8dba0248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:23.157258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29680e-cf1f-42ff-a2e0-19aa8dba0248 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:23.157735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29680e-cf1f-42ff-a2e0-19aa8dba0248 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:23.158121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29680e-cf1f-42ff-a2e0-19aa8dba0248 admin admin] PATH_INFO: `/v3/projects/205fc43a29834dd3bdf4265d0e4aa5c2/users/d5f0141b22b34e3194527d05d33c79cf/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:23.217924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29680e-cf1f-42ff-a2e0-19aa8dba0248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=205fc43a29834dd3bdf4265d0e4aa5c2, user_id=d5f0141b22b34e3194527d05d33c79cf, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:23.225118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29680e-cf1f-42ff-a2e0-19aa8dba0248 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:23.266394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 886/1757] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:23 2022] PUT /identity/v3/projects/205fc43a29834dd3bdf4265d0e4aa5c2/users/d5f0141b22b34e3194527d05d33c79cf/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:23.270356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241e3f-42bc-4934-a84f-22ad18d7b85c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:23.299260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e241e3f-42bc-4934-a84f-22ad18d7b85c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:23.550723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06c451-e8fd-4f86-8d66-88ae49019a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:23.551413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06c451-e8fd-4f86-8d66-88ae49019a00 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:23.551703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06c451-e8fd-4f86-8d66-88ae49019a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:23.551991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06c451-e8fd-4f86-8d66-88ae49019a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:23.552642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06c451-e8fd-4f86-8d66-88ae49019a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:23.556681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06c451-e8fd-4f86-8d66-88ae49019a00 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:23.585691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 872/1758] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:23.588994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29680e-cf1f-42ff-a2e0-19aa8dba0248 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:23.620803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29680e-cf1f-42ff-a2e0-19aa8dba0248 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:23.635604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22dd28-7e8c-4dfd-90c1-c256b18a217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:23.636310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22dd28-7e8c-4dfd-90c1-c256b18a217e admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:23.636602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22dd28-7e8c-4dfd-90c1-c256b18a217e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:23.636884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22dd28-7e8c-4dfd-90c1-c256b18a217e admin admin] PATH_INFO: `/v3/projects/205fc43a29834dd3bdf4265d0e4aa5c2/users/d5f0141b22b34e3194527d05d33c79cf/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:23.640719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22dd28-7e8c-4dfd-90c1-c256b18a217e admin admin] RBAC: Authorizing `identity:create_grant(project_id=205fc43a29834dd3bdf4265d0e4aa5c2, user_id=d5f0141b22b34e3194527d05d33c79cf, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:23.651011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22dd28-7e8c-4dfd-90c1-c256b18a217e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:23.655855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:23.683596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 887/1759] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:23 2022] PUT /identity/v3/projects/205fc43a29834dd3bdf4265d0e4aa5c2/users/d5f0141b22b34e3194527d05d33c79cf/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:23.697196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09defb52-20a1-486e-a6ac-0d65e98b0594 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:23.697812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09defb52-20a1-486e-a6ac-0d65e98b0594 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:23.697921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09defb52-20a1-486e-a6ac-0d65e98b0594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:23.765114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-09defb52-20a1-486e-a6ac-0d65e98b0594 None None] MFA Rules not processed for user `d5f0141b22b34e3194527d05d33c79cf`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:23.766664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-09defb52-20a1-486e-a6ac-0d65e98b0594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:23.847211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:23.944712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 873/1760] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:23.963147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22dd28-7e8c-4dfd-90c1-c256b18a217e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:24.125682 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22dd28-7e8c-4dfd-90c1-c256b18a217e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:24.149956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf8fb0-0eb9-4b2b-8126-e428bc0427a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:24.150936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf8fb0-0eb9-4b2b-8126-e428bc0427a7 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:24.151404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf8fb0-0eb9-4b2b-8126-e428bc0427a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:24.152174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf8fb0-0eb9-4b2b-8126-e428bc0427a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:24.195220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf8fb0-0eb9-4b2b-8126-e428bc0427a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:24.196704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf8fb0-0eb9-4b2b-8126-e428bc0427a7 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:24.222469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 888/1761] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:25.479532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d940-b7d6-429c-a88d-1c6d2befdeed None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:25.481173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d940-b7d6-429c-a88d-1c6d2befdeed None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:25.481837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d940-b7d6-429c-a88d-1c6d2befdeed None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:25.484467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 874/1762] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:49:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:25.563676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf8fb0-0eb9-4b2b-8126-e428bc0427a7 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:25.582271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf8fb0-0eb9-4b2b-8126-e428bc0427a7 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:25.594087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094f1da-7b26-4b83-a474-86378996f798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:25.594812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f1da-7b26-4b83-a474-86378996f798 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:25.595162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f1da-7b26-4b83-a474-86378996f798 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:25.595675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094f1da-7b26-4b83-a474-86378996f798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:25.599351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1094f1da-7b26-4b83-a474-86378996f798 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:25.628275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd702-e0b4-438d-b542-be490e27e117 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:25.629036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd702-e0b4-438d-b542-be490e27e117 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:25.629835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd702-e0b4-438d-b542-be490e27e117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:25.692493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b69dd702-e0b4-438d-b542-be490e27e117 None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:25.695568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b69dd702-e0b4-438d-b542-be490e27e117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:25.817793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094f1da-7b26-4b83-a474-86378996f798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:25.833686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094f1da-7b26-4b83-a474-86378996f798 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:25.866308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 889/1763] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:25.915063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07344cef-5263-4314-b003-98b0c7ea71e1 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:25.915498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07344cef-5263-4314-b003-98b0c7ea71e1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:25.915936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07344cef-5263-4314-b003-98b0c7ea71e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:25.980812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-07344cef-5263-4314-b003-98b0c7ea71e1 None None] MFA Rules not processed for user `3177776f47ef4d3e937c2e8efb360e79`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:25.982960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-07344cef-5263-4314-b003-98b0c7ea71e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:26.005970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 875/1764] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:49:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:26.156566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 890/1765] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:26.355657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd702-e0b4-438d-b542-be490e27e117 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:26.374151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd702-e0b4-438d-b542-be490e27e117 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:26.392451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:26.393441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:26.393764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:26.396332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:26.440643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:26.445257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:26.525166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 876/1766] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:26.556253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:26.559086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:26.559757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07344cef-5263-4314-b003-98b0c7ea71e1 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:26.561437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-07344cef-5263-4314-b003-98b0c7ea71e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:26.593186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07344cef-5263-4314-b003-98b0c7ea71e1 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:26.605757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ac0f9-4bbb-45d0-8837-fad18f22d515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:26.606497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ac0f9-4bbb-45d0-8837-fad18f22d515 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:26.606832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ac0f9-4bbb-45d0-8837-fad18f22d515 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:26.607350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ac0f9-4bbb-45d0-8837-fad18f22d515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:26.642750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ac0f9-4bbb-45d0-8837-fad18f22d515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:26.646751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ac0f9-4bbb-45d0-8837-fad18f22d515 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:26.671586 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 891/1767] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:26.824167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb843b3-ecea-49c1-86f5-90ea2f0d1692 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:26.850973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:26.851567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:26.851845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:26.852117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] PATH_INFO: `/v3/projects/ae6f32eae82d4730ad4a0a5585ff4235` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:26.854488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6f32eae82d4730ad4a0a5585ff4235)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:26.856973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:26.893892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ac0f9-4bbb-45d0-8837-fad18f22d515 service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:26.929152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ac0f9-4bbb-45d0-8837-fad18f22d515 service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:26.934191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6f32eae82d4730ad4a0a5585ff4235', 'request_id': 'req-fb5678f2-5aa8-4875-a706-b3860a3e1d38'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:26.940139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:26.949230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:26.952219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6f32eae82d4730ad4a0a5585ff4235', 'request_id': 'req-fb5678f2-5aa8-4875-a706-b3860a3e1d38'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:26.966620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6f32eae82d4730ad4a0a5585ff4235', 'request_id': 'req-fb5678f2-5aa8-4875-a706-b3860a3e1d38'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.006887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] The token cache is being invalidate because project ae6f32eae82d4730ad4a0a5585ff4235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:27.007189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.018711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.022888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 877/1768] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:26 2022] DELETE /identity/v3/projects/ae6f32eae82d4730ad4a0a5585ff4235 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:27.027513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:27.032237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:27.085235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:27.086863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:27.087632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:27.087913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] PATH_INFO: `/v3/users/f638a8db83b84cd580aa3f81327bc719` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:27.091320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f638a8db83b84cd580aa3f81327bc719)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:27.096175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:27.179306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5678f2-5aa8-4875-a706-b3860a3e1d38 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:27.185041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:27.185780 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:27.186065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:27.186341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] PATH_INFO: `/v3/users/eada5dcab7134f2cbdf74e924c59db94` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:27.187700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eada5dcab7134f2cbdf74e924c59db94)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:27.196684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f638a8db83b84cd580aa3f81327bc719'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.202592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:27.222668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f638a8db83b84cd580aa3f81327bc719'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.223358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f638a8db83b84cd580aa3f81327bc719'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.258975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f638a8db83b84cd580aa3f81327bc719'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.283784 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e646741-677d-4d58-b515-52daa433797f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f638a8db83b84cd580aa3f81327bc719'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.293658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eada5dcab7134f2cbdf74e924c59db94'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.308640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 892/1769] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:26 2022] DELETE /identity/v3/users/f638a8db83b84cd580aa3f81327bc719 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:27.313175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd78e0-d9fd-4522-900e-80a010d589fb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:27.313528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd78e0-d9fd-4522-900e-80a010d589fb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:27.313834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd78e0-d9fd-4522-900e-80a010d589fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:27.318106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eada5dcab7134f2cbdf74e924c59db94'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.319764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eada5dcab7134f2cbdf74e924c59db94'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.343816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eada5dcab7134f2cbdf74e924c59db94'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.354160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eada5dcab7134f2cbdf74e924c59db94'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:27.360708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-24dd78e0-d9fd-4522-900e-80a010d589fb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:27.361963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-24dd78e0-d9fd-4522-900e-80a010d589fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:27.365050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 878/1770] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:27 2022] DELETE /identity/v3/users/eada5dcab7134f2cbdf74e924c59db94 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:27.501054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 893/1771] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:27.571311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:27.572949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:27.731515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5440f-fe60-4c62-95c6-cdb5fd5b6d2c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:27.745129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:27.747329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:27.747812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:27.748285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:27.763946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:27.765857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:27.814960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 879/1772] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:28.796037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:28.797525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0cd5c-8aea-4def-a8f6-f82762d0e2d9 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:28.798150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:28.799536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0cd5c-8aea-4def-a8f6-f82762d0e2d9 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:28.803116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0cd5c-8aea-4def-a8f6-f82762d0e2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:28.811980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:28.825375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0a977-1c88-4fb1-88e4-c01c5ad213af service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:28.833522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:28.835680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:28.837665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:28.839396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] PATH_INFO: `/v3/projects/f5fce0c227ec4bd08c780f97a2297597` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:28.844911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5fce0c227ec4bd08c780f97a2297597)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:28.849119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:28.871721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-78f0cd5c-8aea-4def-a8f6-f82762d0e2d9 None None] MFA Rules not processed for user `15069aa932aa473983c4102961b855d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:28.874029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-78f0cd5c-8aea-4def-a8f6-f82762d0e2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:28.933849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:28.987726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 894/1773] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:28.991701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5fce0c227ec4bd08c780f97a2297597', 'request_id': 'req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:29.011722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5fce0c227ec4bd08c780f97a2297597', 'request_id': 'req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:29.031554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5fce0c227ec4bd08c780f97a2297597', 'request_id': 'req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:29.054954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] The token cache is being invalidate because project f5fce0c227ec4bd08c780f97a2297597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:29.055256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:29.055979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d97f2f6-03b5-41d3-a69e-d92cced0b2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:29.062358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 880/1774] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:28 2022] DELETE /identity/v3/projects/f5fce0c227ec4bd08c780f97a2297597 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:29.063597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d8068-7591-43f4-9a99-2c0de2f9210a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:29.064554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d8068-7591-43f4-9a99-2c0de2f9210a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:29.064985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d8068-7591-43f4-9a99-2c0de2f9210a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:29.097584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-429d8068-7591-43f4-9a99-2c0de2f9210a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:29.099042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-429d8068-7591-43f4-9a99-2c0de2f9210a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:29.193031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 895/1775] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:29.201831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f1cd-9aff-4a79-be25-3e2d780ef53b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:29.202120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f1cd-9aff-4a79-be25-3e2d780ef53b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:29.202398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f1cd-9aff-4a79-be25-3e2d780ef53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:29.270799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2866f1cd-9aff-4a79-be25-3e2d780ef53b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:29.273062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2866f1cd-9aff-4a79-be25-3e2d780ef53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:29.316595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 881/1776] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:29.317889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d8068-7591-43f4-9a99-2c0de2f9210a None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:29.337443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d8068-7591-43f4-9a99-2c0de2f9210a None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:29.349001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:29.352135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:29.352695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:29.353065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:29.353879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:29.355651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:29.358612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:29.394706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 896/1777] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:49:29.403361 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f1cd-9aff-4a79-be25-3e2d780ef53b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:29.420007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f1cd-9aff-4a79-be25-3e2d780ef53b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:29.429802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:29.431827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:29.433432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:29.433735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:29.435145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:29.442579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:29.589549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 882/1778] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:29 2022] POST /identity/v3/projects => generated 367 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:29.593103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:29.628348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022002a-e556-4e88-b1dd-fbca1892d0c5 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:29.696645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:29.699265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:29.767840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e74799-257c-4797-bf6f-2068796b013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:29.768561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74799-257c-4797-bf6f-2068796b013a admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:29.768915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74799-257c-4797-bf6f-2068796b013a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:29.769263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e74799-257c-4797-bf6f-2068796b013a admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:29.770010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e74799-257c-4797-bf6f-2068796b013a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:29.772820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e74799-257c-4797-bf6f-2068796b013a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:29.814327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 897/1779] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:29 2022] POST /identity/v3/users => generated 284 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:29.818427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e74799-257c-4797-bf6f-2068796b013a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:29.843002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e74799-257c-4797-bf6f-2068796b013a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:29.850835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d5b77-a986-4b31-840b-bf05d0911ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:29.852440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d5b77-a986-4b31-840b-bf05d0911ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:29.852895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d5b77-a986-4b31-840b-bf05d0911ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:29.853347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d5b77-a986-4b31-840b-bf05d0911ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:29.854489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d5b77-a986-4b31-840b-bf05d0911ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:29.856553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d5b77-a986-4b31-840b-bf05d0911ae1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:29.878152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 898/1780] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:49:29.882345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d5b77-a986-4b31-840b-bf05d0911ae1 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:29.910578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d5b77-a986-4b31-840b-bf05d0911ae1 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:29.916457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae661e1-35ea-4ce9-b47e-4d75e071e43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:29.917254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae661e1-35ea-4ce9-b47e-4d75e071e43d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:29.917666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae661e1-35ea-4ce9-b47e-4d75e071e43d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:29.918047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae661e1-35ea-4ce9-b47e-4d75e071e43d admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:29.918833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae661e1-35ea-4ce9-b47e-4d75e071e43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:29.920824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae661e1-35ea-4ce9-b47e-4d75e071e43d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:29.936938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 899/1781] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:29.942781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae661e1-35ea-4ce9-b47e-4d75e071e43d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:29.999575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae661e1-35ea-4ce9-b47e-4d75e071e43d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:29.999904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d3cc0-befe-4b32-85b3-5173ddf13086 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:30.006524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.008750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.009339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.009779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] PATH_INFO: `/v3/projects/9eb68c7843e44d3a93efcbdf9b9091e8` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.015186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb68c7843e44d3a93efcbdf9b9091e8)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.015186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.024201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10bb66-3eeb-438a-9986-95acc84368ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.024980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10bb66-3eeb-438a-9986-95acc84368ab admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.026696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10bb66-3eeb-438a-9986-95acc84368ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.026951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10bb66-3eeb-438a-9986-95acc84368ab admin admin] PATH_INFO: `/v3/projects/dbf04ddadf5e4b0192adaccd3db40b9f/users/c4e90bb30f344c9791035c25d7fc46dc/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.072497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10bb66-3eeb-438a-9986-95acc84368ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf04ddadf5e4b0192adaccd3db40b9f, user_id=c4e90bb30f344c9791035c25d7fc46dc, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.081763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10bb66-3eeb-438a-9986-95acc84368ab admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.119657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb68c7843e44d3a93efcbdf9b9091e8', 'request_id': 'req-049e4096-27e0-4f1f-aa9e-7db842b495d4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:30.127303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb68c7843e44d3a93efcbdf9b9091e8', 'request_id': 'req-049e4096-27e0-4f1f-aa9e-7db842b495d4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:30.129528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:30.136195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb68c7843e44d3a93efcbdf9b9091e8', 'request_id': 'req-049e4096-27e0-4f1f-aa9e-7db842b495d4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:30.142307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 900/1782] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:29 2022] PUT /identity/v3/projects/dbf04ddadf5e4b0192adaccd3db40b9f/users/c4e90bb30f344c9791035c25d7fc46dc/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:30.144691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1f169-3a53-46a5-9854-3c1666650574 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.145008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1f169-3a53-46a5-9854-3c1666650574 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.145468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1f169-3a53-46a5-9854-3c1666650574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.160002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] The token cache is being invalidate because project 9eb68c7843e44d3a93efcbdf9b9091e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:30.160471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:30.161302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-049e4096-27e0-4f1f-aa9e-7db842b495d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:30.164515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 883/1783] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:29 2022] DELETE /identity/v3/projects/9eb68c7843e44d3a93efcbdf9b9091e8 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:30.170468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985f1fb-9835-4bbd-aedd-c2f463e39116 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.170839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985f1fb-9835-4bbd-aedd-c2f463e39116 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.171188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985f1fb-9835-4bbd-aedd-c2f463e39116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.175197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:30.208290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-89a1f169-3a53-46a5-9854-3c1666650574 None None] MFA Rules not processed for user `c4e90bb30f344c9791035c25d7fc46dc`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:30.211305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-89a1f169-3a53-46a5-9854-3c1666650574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:30.224402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1985f1fb-9835-4bbd-aedd-c2f463e39116 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:30.226418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1985f1fb-9835-4bbd-aedd-c2f463e39116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:30.298837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 901/1784] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:30.305104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1f169-3a53-46a5-9854-3c1666650574 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:30.306754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-89a1f169-3a53-46a5-9854-3c1666650574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:30.329798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 884/1785] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:30.338296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430d477-d8e3-4383-9dec-5b6e8011fed0 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.338653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430d477-d8e3-4383-9dec-5b6e8011fed0 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.338939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430d477-d8e3-4383-9dec-5b6e8011fed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.391507 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c430d477-d8e3-4383-9dec-5b6e8011fed0 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:30.395970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c430d477-d8e3-4383-9dec-5b6e8011fed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:30.415038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1f169-3a53-46a5-9854-3c1666650574 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:30.420728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41ba12-50f7-409c-a4e3-09d08f368753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.421436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ba12-50f7-409c-a4e3-09d08f368753 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.422775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ba12-50f7-409c-a4e3-09d08f368753 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.423359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ba12-50f7-409c-a4e3-09d08f368753 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.424077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41ba12-50f7-409c-a4e3-09d08f368753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.426884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 885/1786] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:30.430479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430d477-d8e3-4383-9dec-5b6e8011fed0 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:30.438390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41ba12-50f7-409c-a4e3-09d08f368753 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.451601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430d477-d8e3-4383-9dec-5b6e8011fed0 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:30.463597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.466518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.466806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.467085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.467938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:30.468248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.473549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.487102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 902/1787] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:30 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:30.489355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41ba12-50f7-409c-a4e3-09d08f368753 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:30.508058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 886/1788] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:30.512204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:30.533908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543fe9b-9f2f-4003-9b3d-a57c20d4d736 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:30.608448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41ba12-50f7-409c-a4e3-09d08f368753 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:30.625877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa50e-4ba8-45b7-b03a-9e9c632fb37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.626573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa50e-4ba8-45b7-b03a-9e9c632fb37b admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.626833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa50e-4ba8-45b7-b03a-9e9c632fb37b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.627040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4fa50e-4ba8-45b7-b03a-9e9c632fb37b admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.627761 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fa50e-4ba8-45b7-b03a-9e9c632fb37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.632283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4fa50e-4ba8-45b7-b03a-9e9c632fb37b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.648990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a51fb-0263-4a5d-ac75-e5a9bc0504de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.649723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a51fb-0263-4a5d-ac75-e5a9bc0504de admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.649951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a51fb-0263-4a5d-ac75-e5a9bc0504de admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.650235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a51fb-0263-4a5d-ac75-e5a9bc0504de admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.650880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a51fb-0263-4a5d-ac75-e5a9bc0504de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.652845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a51fb-0263-4a5d-ac75-e5a9bc0504de admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.688086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 903/1789] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:30 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:30.693846 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa50e-4ba8-45b7-b03a-9e9c632fb37b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:30.729158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4fa50e-4ba8-45b7-b03a-9e9c632fb37b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:30.730182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 887/1790] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:30 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:30.735762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a51fb-0263-4a5d-ac75-e5a9bc0504de admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:30.748044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:30.831699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0f862-0e2a-4a5a-b310-5476135f5dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.832280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f862-0e2a-4a5a-b310-5476135f5dee admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.832563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f862-0e2a-4a5a-b310-5476135f5dee admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.836107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f862-0e2a-4a5a-b310-5476135f5dee admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.836823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f862-0e2a-4a5a-b310-5476135f5dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.841566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f862-0e2a-4a5a-b310-5476135f5dee admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.874166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 904/1791] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:30 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:30.875825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0f862-0e2a-4a5a-b310-5476135f5dee admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:30.887821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0f862-0e2a-4a5a-b310-5476135f5dee admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:30.892956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89eba8-38e4-4c7b-a382-4426628f7243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.894916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89eba8-38e4-4c7b-a382-4426628f7243 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.895485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89eba8-38e4-4c7b-a382-4426628f7243 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.895748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89eba8-38e4-4c7b-a382-4426628f7243 admin admin] PATH_INFO: `/v3/projects/9563ec8e976049e8b903d3d8f48ff2b5/users/460cc232b1954980b028c8981cda2e7d/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.921061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a51fb-0263-4a5d-ac75-e5a9bc0504de admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:30.922229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:30.929837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5e95-c169-46b7-8f71-49239ec71ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:30.931122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5e95-c169-46b7-8f71-49239ec71ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:30.931601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5e95-c169-46b7-8f71-49239ec71ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:30.932134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e5e95-c169-46b7-8f71-49239ec71ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:30.933810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e5e95-c169-46b7-8f71-49239ec71ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.936777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e5e95-c169-46b7-8f71-49239ec71ad9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.961338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89eba8-38e4-4c7b-a382-4426628f7243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9563ec8e976049e8b903d3d8f48ff2b5, user_id=460cc232b1954980b028c8981cda2e7d, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:30.968151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89eba8-38e4-4c7b-a382-4426628f7243 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:30.998272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 888/1792] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:30 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:31.005171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5e95-c169-46b7-8f71-49239ec71ad9 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:31.021733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 905/1793] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:30 2022] PUT /identity/v3/projects/9563ec8e976049e8b903d3d8f48ff2b5/users/460cc232b1954980b028c8981cda2e7d/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:31.022915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89eba8-38e4-4c7b-a382-4426628f7243 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:31.057085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e5e95-c169-46b7-8f71-49239ec71ad9 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:31.184225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89eba8-38e4-4c7b-a382-4426628f7243 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:31.190145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a235907-8330-4d26-aa67-487fc57a22a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:31.190825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a235907-8330-4d26-aa67-487fc57a22a4 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.191106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a235907-8330-4d26-aa67-487fc57a22a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.192174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a235907-8330-4d26-aa67-487fc57a22a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.192851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a235907-8330-4d26-aa67-487fc57a22a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:31.195841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a235907-8330-4d26-aa67-487fc57a22a4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:31.210565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 906/1794] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:31.214455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a235907-8330-4d26-aa67-487fc57a22a4 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:31.235720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a235907-8330-4d26-aa67-487fc57a22a4 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:31.242770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:31.242960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.243188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.243465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] PATH_INFO: `/v3/projects/9563ec8e976049e8b903d3d8f48ff2b5/users/460cc232b1954980b028c8981cda2e7d/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.246051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9563ec8e976049e8b903d3d8f48ff2b5, user_id=460cc232b1954980b028c8981cda2e7d, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:31.249523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:31.277018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 907/1795] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:31 2022] PUT /identity/v3/projects/9563ec8e976049e8b903d3d8f48ff2b5/users/460cc232b1954980b028c8981cda2e7d/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:31.277018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:31.280776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:31.286768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518ec1a-0b9f-4a80-97f4-cabb8fee29a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:31.287467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ec1a-0b9f-4a80-97f4-cabb8fee29a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.287758 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ec1a-0b9f-4a80-97f4-cabb8fee29a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.288034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ec1a-0b9f-4a80-97f4-cabb8fee29a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.288819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518ec1a-0b9f-4a80-97f4-cabb8fee29a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:31.291078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518ec1a-0b9f-4a80-97f4-cabb8fee29a7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:31.314025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 889/1796] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:31.315361 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54778-13b7-436b-83f1-1af34c5f4bbc None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.315651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54778-13b7-436b-83f1-1af34c5f4bbc None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.315927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf54778-13b7-436b-83f1-1af34c5f4bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.379188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755267e5-280a-48be-aefa-279aaf637b41 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:31.384512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:31.385231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.385686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.385873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.386543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-9cf54778-13b7-436b-83f1-1af34c5f4bbc None None] MFA Rules not processed for user `460cc232b1954980b028c8981cda2e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:31.387716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:31.388980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9cf54778-13b7-436b-83f1-1af34c5f4bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:31.417782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:31.478391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:31.481088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:31.493911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 908/1797] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:31.494860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:31.577742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 890/1798] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:31.589782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54778-13b7-436b-83f1-1af34c5f4bbc None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:31.591371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9cf54778-13b7-436b-83f1-1af34c5f4bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:31.596416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825939a-8f5b-4c3a-acbe-b2d561013236 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:31.604111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0416a4-4293-4a21-9315-150e5a67f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:31.604111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0416a4-4293-4a21-9315-150e5a67f99d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.604111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0416a4-4293-4a21-9315-150e5a67f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.604111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0416a4-4293-4a21-9315-150e5a67f99d admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.604538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0416a4-4293-4a21-9315-150e5a67f99d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:31.606091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0416a4-4293-4a21-9315-150e5a67f99d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:31.619055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 909/1799] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:31.624759 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0416a4-4293-4a21-9315-150e5a67f99d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:31.633191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf54778-13b7-436b-83f1-1af34c5f4bbc None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:31.639066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0416a4-4293-4a21-9315-150e5a67f99d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:31.645427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d698bf-3f21-45b9-b9f8-68e16f92dd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:31.648183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d698bf-3f21-45b9-b9f8-68e16f92dd9b admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.648457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2270c4a-30ce-4900-ae2f-7263b0e6d9da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:31.648954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d698bf-3f21-45b9-b9f8-68e16f92dd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.649059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270c4a-30ce-4900-ae2f-7263b0e6d9da service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.649446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270c4a-30ce-4900-ae2f-7263b0e6d9da service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.649776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2270c4a-30ce-4900-ae2f-7263b0e6d9da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.649994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d698bf-3f21-45b9-b9f8-68e16f92dd9b admin admin] PATH_INFO: `/v3/projects/55caf1cbd6d3425998c08f9268690c04/users/7f77ff80fc3649ea99a093b9bc6e34af/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.679602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d698bf-3f21-45b9-b9f8-68e16f92dd9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=55caf1cbd6d3425998c08f9268690c04, user_id=7f77ff80fc3649ea99a093b9bc6e34af, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:31.682820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d698bf-3f21-45b9-b9f8-68e16f92dd9b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:31.689861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2270c4a-30ce-4900-ae2f-7263b0e6d9da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:31.691997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2270c4a-30ce-4900-ae2f-7263b0e6d9da service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:31.707705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 910/1800] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:31 2022] PUT /identity/v3/projects/55caf1cbd6d3425998c08f9268690c04/users/7f77ff80fc3649ea99a093b9bc6e34af/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:31.719338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f71f26-3659-4a4a-a5d9-2a4491b1f8d1 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.719912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f71f26-3659-4a4a-a5d9-2a4491b1f8d1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.721926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f71f26-3659-4a4a-a5d9-2a4491b1f8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.724336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 891/1801] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:31.787516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-43f71f26-3659-4a4a-a5d9-2a4491b1f8d1 None None] MFA Rules not processed for user `7f77ff80fc3649ea99a093b9bc6e34af`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:31.789701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-43f71f26-3659-4a4a-a5d9-2a4491b1f8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:31.822437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2270c4a-30ce-4900-ae2f-7263b0e6d9da service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:31.835732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2270c4a-30ce-4900-ae2f-7263b0e6d9da service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:31.940160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 911/1802] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:31.945276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69889736-aad9-40bd-9603-655a3e186b28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:31.945941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889736-aad9-40bd-9603-655a3e186b28 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:31.946454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889736-aad9-40bd-9603-655a3e186b28 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:31.946454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69889736-aad9-40bd-9603-655a3e186b28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:31.959452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:32.061453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69889736-aad9-40bd-9603-655a3e186b28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:32.063261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69889736-aad9-40bd-9603-655a3e186b28 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:32.080531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 892/1803] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:32.205098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6e88b-de94-4628-bebd-913efa043a6f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:32.205388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6e88b-de94-4628-bebd-913efa043a6f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:32.205723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6e88b-de94-4628-bebd-913efa043a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:32.245752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4ab6e88b-de94-4628-bebd-913efa043a6f None None] MFA Rules not processed for user `61b9ae7499154e5eab4f3c23d8bb6785`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:32.249297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4ab6e88b-de94-4628-bebd-913efa043a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:32.339845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:32.522562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 912/1804] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:32.607248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69889736-aad9-40bd-9603-655a3e186b28 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:32.630069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69889736-aad9-40bd-9603-655a3e186b28 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:32.633828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dea1-0a5a-40a7-b38c-e10efb72d895 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:32.634111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dea1-0a5a-40a7-b38c-e10efb72d895 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:32.634380 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465dea1-0a5a-40a7-b38c-e10efb72d895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:32.638554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7a1e7-3469-40ac-a316-ce8a452def9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:32.639273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7a1e7-3469-40ac-a316-ce8a452def9b service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:32.639561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7a1e7-3469-40ac-a316-ce8a452def9b service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:32.639838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7a1e7-3469-40ac-a316-ce8a452def9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:32.667271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5465dea1-0a5a-40a7-b38c-e10efb72d895 None None] MFA Rules not processed for user `3177776f47ef4d3e937c2e8efb360e79`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:32.668492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5465dea1-0a5a-40a7-b38c-e10efb72d895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:32.692023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7a1e7-3469-40ac-a316-ce8a452def9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:32.694177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7a1e7-3469-40ac-a316-ce8a452def9b service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:32.710676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 893/1805] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:32.762337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 913/1806] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:34.412048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65aea9-6424-4ab1-a1ac-8f3b76c2ebac None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:34.413313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65aea9-6424-4ab1-a1ac-8f3b76c2ebac None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:34.414128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65aea9-6424-4ab1-a1ac-8f3b76c2ebac None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:34.419480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 894/1807] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:49:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:49:34.430402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d173-d1db-4a95-b4fc-4fc49273be7f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:34.430886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d173-d1db-4a95-b4fc-4fc49273be7f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:34.430958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f7d173-d1db-4a95-b4fc-4fc49273be7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:34.466270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-93f7d173-d1db-4a95-b4fc-4fc49273be7f None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:34.467499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-93f7d173-d1db-4a95-b4fc-4fc49273be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:34.692150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 914/1808] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:49:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:35.226867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65aea9-6424-4ab1-a1ac-8f3b76c2ebac None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:35.470367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65aea9-6424-4ab1-a1ac-8f3b76c2ebac None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:35.483796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:35.484644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:35.484990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:35.485376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] PATH_INFO: `/v3/users/7f77ff80fc3649ea99a093b9bc6e34af` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:35.487727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f77ff80fc3649ea99a093b9bc6e34af)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:35.492637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:35.498939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d173-d1db-4a95-b4fc-4fc49273be7f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:35.502433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-93f7d173-d1db-4a95-b4fc-4fc49273be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:35.547297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f7d173-d1db-4a95-b4fc-4fc49273be7f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:35.577932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593a4d1-385e-4aa3-be65-085e5223dacd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:35.579156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a4d1-385e-4aa3-be65-085e5223dacd service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:35.579663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a4d1-385e-4aa3-be65-085e5223dacd service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:35.580142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a4d1-385e-4aa3-be65-085e5223dacd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:35.610049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593a4d1-385e-4aa3-be65-085e5223dacd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:35.616441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593a4d1-385e-4aa3-be65-085e5223dacd service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:35.657936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 915/1809] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:35.662926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f77ff80fc3649ea99a093b9bc6e34af'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:35.677802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f77ff80fc3649ea99a093b9bc6e34af'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:35.679067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f77ff80fc3649ea99a093b9bc6e34af'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:35.727420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f77ff80fc3649ea99a093b9bc6e34af'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:35.760050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f77ff80fc3649ea99a093b9bc6e34af'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:35.785654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 895/1810] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:35 2022] DELETE /identity/v3/users/7f77ff80fc3649ea99a093b9bc6e34af => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:35.790464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a78f-dc3c-46a2-9201-e93f2d0c9cef None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:35.790761 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a78f-dc3c-46a2-9201-e93f2d0c9cef None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:35.791135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81a78f-dc3c-46a2-9201-e93f2d0c9cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:35.817761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:35.818822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:35.863824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:35.869931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ce81a78f-dc3c-46a2-9201-e93f2d0c9cef None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:35.871289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ce81a78f-dc3c-46a2-9201-e93f2d0c9cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:36.016727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:36.055751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b15791-694d-4622-8541-3f06bac184cf admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:36.078567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:36.079891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:36.080278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 916/1811] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:36.080559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:36.081065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] PATH_INFO: `/v3/users/a09ffb27443141bb97270189876ded74` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:36.083884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09ffb27443141bb97270189876ded74)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:36.088630 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:36.129535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81a78f-dc3c-46a2-9201-e93f2d0c9cef None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:36.131607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ce81a78f-dc3c-46a2-9201-e93f2d0c9cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:36.205523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09ffb27443141bb97270189876ded74'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.222414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09ffb27443141bb97270189876ded74'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.223068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09ffb27443141bb97270189876ded74'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.266167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09ffb27443141bb97270189876ded74'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.284070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-473fb172-b6fd-40b3-9c5d-bca7bd52c1be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09ffb27443141bb97270189876ded74'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.288667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81a78f-dc3c-46a2-9201-e93f2d0c9cef None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:36.295250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:36.296068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:36.296642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:36.298413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 896/1812] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:35 2022] DELETE /identity/v3/users/a09ffb27443141bb97270189876ded74 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:36.303580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b195-5b50-4527-b248-ce7b9bc2f503 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:36.303729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b195-5b50-4527-b248-ce7b9bc2f503 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:36.303934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0b195-5b50-4527-b248-ce7b9bc2f503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:36.317644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:36.322830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:36.351019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-40f0b195-5b50-4527-b248-ce7b9bc2f503 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:36.355639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-40f0b195-5b50-4527-b248-ce7b9bc2f503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:36.374027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:36.377643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:36.405512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 917/1813] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:36.503394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 897/1814] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:36.509555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:36.511152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:36.525866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0b195-5b50-4527-b248-ce7b9bc2f503 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:36.527901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-40f0b195-5b50-4527-b248-ce7b9bc2f503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:36.536365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d161e-ef18-47c2-b8a9-7520edb7d088 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:36.544488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:36.545429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:36.545881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:36.546428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] PATH_INFO: `/v3/users/c4e90bb30f344c9791035c25d7fc46dc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:36.548059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e90bb30f344c9791035c25d7fc46dc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:36.551540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:36.651535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:36.691080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0b195-5b50-4527-b248-ce7b9bc2f503 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:36.700112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c073ad-d1ed-4407-a420-7da8b6c4f8db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:36.700733 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c073ad-d1ed-4407-a420-7da8b6c4f8db service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:36.701015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c073ad-d1ed-4407-a420-7da8b6c4f8db service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:36.701298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c073ad-d1ed-4407-a420-7da8b6c4f8db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:36.710879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e90bb30f344c9791035c25d7fc46dc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.721951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c073ad-d1ed-4407-a420-7da8b6c4f8db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:36.725320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e90bb30f344c9791035c25d7fc46dc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.726674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e90bb30f344c9791035c25d7fc46dc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.728033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c073ad-d1ed-4407-a420-7da8b6c4f8db service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:36.762044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e90bb30f344c9791035c25d7fc46dc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.785316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e90bb30f344c9791035c25d7fc46dc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:36.802804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 898/1815] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:36.803328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 918/1816] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:36 2022] DELETE /identity/v3/users/c4e90bb30f344c9791035c25d7fc46dc => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:36.808054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c7f37-691d-4b34-b283-53dbc7946e6f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:36.808342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c7f37-691d-4b34-b283-53dbc7946e6f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:36.808621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c7f37-691d-4b34-b283-53dbc7946e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:36.868976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-bb3c7f37-691d-4b34-b283-53dbc7946e6f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:36.870888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bb3c7f37-691d-4b34-b283-53dbc7946e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:36.975920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 899/1817] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:37.005448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:37.008475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:37.102530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4791eff-02f4-4b61-992d-b96cde63b22b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:37.107892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:37.108891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:37.109727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:37.110169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:37.138009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:37.141705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:37.157264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 919/1818] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:38.426591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c7f37-691d-4b34-b283-53dbc7946e6f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:38.428797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bb3c7f37-691d-4b34-b283-53dbc7946e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:38.455510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c7f37-691d-4b34-b283-53dbc7946e6f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:38.466413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:38.468794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:38.470838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:38.472704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] PATH_INFO: `/v3/projects/37fa5f5c8f9740689c1434c673920fdf` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:38.474287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37fa5f5c8f9740689c1434c673920fdf)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:38.482022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:38.530081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:38.531998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:38.564281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76db7-25c2-409d-9409-73e468148046 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:38.582549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:38.585165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:38.585630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:38.586470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] PATH_INFO: `/v3/projects/55caf1cbd6d3425998c08f9268690c04` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:38.590075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] RBAC: Authorizing `identity:delete_project(project_id=55caf1cbd6d3425998c08f9268690c04)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:38.605061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:38.615240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:38.643447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37fa5f5c8f9740689c1434c673920fdf', 'request_id': 'req-301b4a9f-f8e7-410c-be31-8c8f32633689'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.655164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37fa5f5c8f9740689c1434c673920fdf', 'request_id': 'req-301b4a9f-f8e7-410c-be31-8c8f32633689'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.668263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37fa5f5c8f9740689c1434c673920fdf', 'request_id': 'req-301b4a9f-f8e7-410c-be31-8c8f32633689'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.701576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] The token cache is being invalidate because project 37fa5f5c8f9740689c1434c673920fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:38.701576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.702356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.705114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 900/1819] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:38 2022] DELETE /identity/v3/projects/37fa5f5c8f9740689c1434c673920fdf => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:38.706129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:38.709341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:38.734010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55caf1cbd6d3425998c08f9268690c04', 'request_id': 'req-e109ada4-f892-40a7-b03e-fac35a86167b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.746681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55caf1cbd6d3425998c08f9268690c04', 'request_id': 'req-e109ada4-f892-40a7-b03e-fac35a86167b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.760702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55caf1cbd6d3425998c08f9268690c04', 'request_id': 'req-e109ada4-f892-40a7-b03e-fac35a86167b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.784697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] The token cache is being invalidate because project 55caf1cbd6d3425998c08f9268690c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:38.785072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.785940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:38.789797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 920/1820] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:38 2022] DELETE /identity/v3/projects/55caf1cbd6d3425998c08f9268690c04 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:38.792289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:38.795242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:38.892213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b4a9f-f8e7-410c-be31-8c8f32633689 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:38.897160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:38.988093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109ada4-f892-40a7-b03e-fac35a86167b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:38.995523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:38.996118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:38.996815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:38.996933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:38.997270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:38.997751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:38.997856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] PATH_INFO: `/v3/users/53a0ee56cb51446ca836587e3c3e7be6` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:38.998406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] PATH_INFO: `/v3/projects/dbf04ddadf5e4b0192adaccd3db40b9f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:38.999060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a0ee56cb51446ca836587e3c3e7be6)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.003345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.004308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf04ddadf5e4b0192adaccd3db40b9f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.015962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.164869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a0ee56cb51446ca836587e3c3e7be6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.175290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a0ee56cb51446ca836587e3c3e7be6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.176184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a0ee56cb51446ca836587e3c3e7be6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.180498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf04ddadf5e4b0192adaccd3db40b9f', 'request_id': 'req-72cb114b-fd2a-468c-8f36-b8ed389c7225'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.188084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf04ddadf5e4b0192adaccd3db40b9f', 'request_id': 'req-72cb114b-fd2a-468c-8f36-b8ed389c7225'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.191745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a0ee56cb51446ca836587e3c3e7be6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.196696 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf04ddadf5e4b0192adaccd3db40b9f', 'request_id': 'req-72cb114b-fd2a-468c-8f36-b8ed389c7225'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.200646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c90d81f2-ce1f-4e2f-a4ab-e2d03179fb9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a0ee56cb51446ca836587e3c3e7be6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.210200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 921/1821] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:38 2022] DELETE /identity/v3/users/53a0ee56cb51446ca836587e3c3e7be6 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:39.211766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21f8a-489b-4832-aa52-53a9d780b37f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.212246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21f8a-489b-4832-aa52-53a9d780b37f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.212540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c21f8a-489b-4832-aa52-53a9d780b37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.214947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] The token cache is being invalidate because project dbf04ddadf5e4b0192adaccd3db40b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:39.215322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.216077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.217875 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 901/1822] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:38 2022] DELETE /identity/v3/projects/dbf04ddadf5e4b0192adaccd3db40b9f => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:39.221956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:39.223464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:39.238233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-53c21f8a-489b-4832-aa52-53a9d780b37f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:39.239889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-53c21f8a-489b-4832-aa52-53a9d780b37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:39.247216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:39.301403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb114b-fd2a-468c-8f36-b8ed389c7225 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:39.319439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 922/1823] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:39.328848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ebae-f45b-4a05-af5d-3f672e6b2140 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.329278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ebae-f45b-4a05-af5d-3f672e6b2140 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.329552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ebae-f45b-4a05-af5d-3f672e6b2140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.354191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9ab5ebae-f45b-4a05-af5d-3f672e6b2140 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:39.354772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:39.355101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9ab5ebae-f45b-4a05-af5d-3f672e6b2140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:39.368472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 923/1824] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:39.370091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5ebae-f45b-4a05-af5d-3f672e6b2140 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:39.375732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:39.377032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.377032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.377032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] PATH_INFO: `/v3/users/460cc232b1954980b028c8981cda2e7d` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.378224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=460cc232b1954980b028c8981cda2e7d)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.379933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5ebae-f45b-4a05-af5d-3f672e6b2140 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:39.380408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.385263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:39.386100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.386542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.387003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.387661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:39.387968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.389698 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.402670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 924/1825] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:39.418735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:39.435012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5bbd1-352e-4b2d-8f25-d15cc32ff1e3 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:39.442503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:39.443160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.443426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.444426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.444643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.447063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.459551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '460cc232b1954980b028c8981cda2e7d'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.468594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '460cc232b1954980b028c8981cda2e7d'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.469413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '460cc232b1954980b028c8981cda2e7d'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.485646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 925/1826] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:39 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:39.487329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:39.489336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:39.494326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '460cc232b1954980b028c8981cda2e7d'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.509174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '460cc232b1954980b028c8981cda2e7d'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:39.521839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 902/1827] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:39 2022] DELETE /identity/v3/users/460cc232b1954980b028c8981cda2e7d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:39.645006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528e507-cd49-4aad-8176-32bbda1b4e66 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:39.770374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13200f53-6e6d-4443-9ea3-992d832841b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:39.771006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13200f53-6e6d-4443-9ea3-992d832841b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.772579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13200f53-6e6d-4443-9ea3-992d832841b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.772879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13200f53-6e6d-4443-9ea3-992d832841b4 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.773727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13200f53-6e6d-4443-9ea3-992d832841b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.777440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13200f53-6e6d-4443-9ea3-992d832841b4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.824703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 926/1828] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:39 2022] POST /identity/v3/users => generated 286 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:39.826373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:39.859272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ada01c-0270-4015-ac3e-8504fd44ab69 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:39.871385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3bb61-cba0-4d11-96d3-91f982983d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:39.872675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3bb61-cba0-4d11-96d3-91f982983d7f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.873296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3bb61-cba0-4d11-96d3-91f982983d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.873752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3bb61-cba0-4d11-96d3-91f982983d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.876008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3bb61-cba0-4d11-96d3-91f982983d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.878465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b3bb61-cba0-4d11-96d3-91f982983d7f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.902904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 903/1829] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:39.903660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13200f53-6e6d-4443-9ea3-992d832841b4 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:39.923070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13200f53-6e6d-4443-9ea3-992d832841b4 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:39.935016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32351c86-82d2-4c89-b3ed-7c851bfa3d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:39.935632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351c86-82d2-4c89-b3ed-7c851bfa3d4a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.935920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351c86-82d2-4c89-b3ed-7c851bfa3d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.936191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32351c86-82d2-4c89-b3ed-7c851bfa3d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:39.937235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351c86-82d2-4c89-b3ed-7c851bfa3d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:39.938475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32351c86-82d2-4c89-b3ed-7c851bfa3d4a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:39.939996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:39.961728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 927/1830] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:39.962552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3bb61-cba0-4d11-96d3-91f982983d7f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:39.976846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b3bb61-cba0-4d11-96d3-91f982983d7f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:39.983567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:39.984269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:39.984548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:39.984824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] PATH_INFO: `/v3/projects/e67b9d2cb28f44c99e5d331f16c8a4d2/users/5b7bacd3026045759540b1b7dd10c8dd/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:40.008751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e67b9d2cb28f44c99e5d331f16c8a4d2, user_id=5b7bacd3026045759540b1b7dd10c8dd, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:40.018183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:40.044127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 904/1831] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:39 2022] PUT /identity/v3/projects/e67b9d2cb28f44c99e5d331f16c8a4d2/users/5b7bacd3026045759540b1b7dd10c8dd/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:40.056651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b482097-f134-4b8d-a821-ea92344109e2 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:40.056945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b482097-f134-4b8d-a821-ea92344109e2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:40.057223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b482097-f134-4b8d-a821-ea92344109e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:40.090637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3b482097-f134-4b8d-a821-ea92344109e2 None None] MFA Rules not processed for user `5b7bacd3026045759540b1b7dd10c8dd`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:40.091888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3b482097-f134-4b8d-a821-ea92344109e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:40.201020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 928/1832] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:40.212925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:40.217429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:40.239864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b482097-f134-4b8d-a821-ea92344109e2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:40.242100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3b482097-f134-4b8d-a821-ea92344109e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:40.411456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b482097-f134-4b8d-a821-ea92344109e2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:40.416948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:40.417630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:40.417918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:40.418194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] PATH_INFO: `/v3/projects/5fc49c89aa8040e1a353944918139d3f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:40.419376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc49c89aa8040e1a353944918139d3f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:40.422047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:40.425560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36bce4-e813-429d-b23c-714d9426f5b2 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:40.449183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995ce21-9d0c-44fa-94b7-98548dc36b54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d78c1e1b6e46e3ba6271e6743d3c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:40.451798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ce21-9d0c-44fa-94b7-98548dc36b54 service glance] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:40.452580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ce21-9d0c-44fa-94b7-98548dc36b54 service glance] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:40.453574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ce21-9d0c-44fa-94b7-98548dc36b54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:40.466615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995ce21-9d0c-44fa-94b7-98548dc36b54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:40.469041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995ce21-9d0c-44fa-94b7-98548dc36b54 service glance] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:40.514962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 905/1833] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:40.524956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc49c89aa8040e1a353944918139d3f', 'request_id': 'req-e715b5b8-899f-4dea-b267-507414bbd564'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.536549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc49c89aa8040e1a353944918139d3f', 'request_id': 'req-e715b5b8-899f-4dea-b267-507414bbd564'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.544308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc49c89aa8040e1a353944918139d3f', 'request_id': 'req-e715b5b8-899f-4dea-b267-507414bbd564'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.566254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] The token cache is being invalidate because project 5fc49c89aa8040e1a353944918139d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:40.567062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.567952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.574393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 929/1834] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:40 2022] DELETE /identity/v3/projects/5fc49c89aa8040e1a353944918139d3f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:40.574996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995ce21-9d0c-44fa-94b7-98548dc36b54 service glance] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:40.592481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995ce21-9d0c-44fa-94b7-98548dc36b54 service glance] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:40.606821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:40.740533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:40.741197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:40.741594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:40.741760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] PATH_INFO: `/v3/users/d5f0141b22b34e3194527d05d33c79cf` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:40.745148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f0141b22b34e3194527d05d33c79cf)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:40.747348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:40.842322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f0141b22b34e3194527d05d33c79cf'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.855334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f0141b22b34e3194527d05d33c79cf'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.856577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f0141b22b34e3194527d05d33c79cf'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.881971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f0141b22b34e3194527d05d33c79cf'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.895734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2dd5464d-e643-4346-baf6-671af88f5c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f0141b22b34e3194527d05d33c79cf'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:40.909944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 906/1835] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:40 2022] DELETE /identity/v3/users/d5f0141b22b34e3194527d05d33c79cf => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:41.189833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:41.191715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:41.214245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e715b5b8-899f-4dea-b267-507414bbd564 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:41.232741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:41.343702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:41.344391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:41.344674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:41.345255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] PATH_INFO: `/v3/projects/9563ec8e976049e8b903d3d8f48ff2b5` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:41.347623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9563ec8e976049e8b903d3d8f48ff2b5)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:41.350352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:41.463924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9563ec8e976049e8b903d3d8f48ff2b5', 'request_id': 'req-1c30345e-d201-456d-a2e1-bb112d95807d'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.470229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9563ec8e976049e8b903d3d8f48ff2b5', 'request_id': 'req-1c30345e-d201-456d-a2e1-bb112d95807d'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.485419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9563ec8e976049e8b903d3d8f48ff2b5', 'request_id': 'req-1c30345e-d201-456d-a2e1-bb112d95807d'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.499961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] The token cache is being invalidate because project 9563ec8e976049e8b903d3d8f48ff2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:41.500292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.500923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1c30345e-d201-456d-a2e1-bb112d95807d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.506612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 930/1836] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:41 2022] DELETE /identity/v3/projects/9563ec8e976049e8b903d3d8f48ff2b5 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:41.509022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b15b-7220-4345-b325-b07edde16238 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:41.509496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b15b-7220-4345-b325-b07edde16238 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:41.509595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1b15b-7220-4345-b325-b07edde16238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:41.543493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b0f1b15b-7220-4345-b325-b07edde16238 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:41.545712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b0f1b15b-7220-4345-b325-b07edde16238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:41.645886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 907/1837] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:41.648845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544146e-16bd-4cfb-a710-b9f79b4ae6b7 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:41.649136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544146e-16bd-4cfb-a710-b9f79b4ae6b7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:41.649413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544146e-16bd-4cfb-a710-b9f79b4ae6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:41.674922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-e544146e-16bd-4cfb-a710-b9f79b4ae6b7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:41.676664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e544146e-16bd-4cfb-a710-b9f79b4ae6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:41.689856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 931/1838] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:41.695050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1b15b-7220-4345-b325-b07edde16238 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:41.712107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1b15b-7220-4345-b325-b07edde16238 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:41.723217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:41.724505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:41.725022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:41.726108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:41.728096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:41.728523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:41.735519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:41.737178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544146e-16bd-4cfb-a710-b9f79b4ae6b7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:41.738942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e544146e-16bd-4cfb-a710-b9f79b4ae6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:41.754512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 908/1839] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:49:41.774716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:41.795891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4f881-7ba2-4968-9744-3e28e4f27686 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:41.800627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544146e-16bd-4cfb-a710-b9f79b4ae6b7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:41.806323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c57bd9-81c9-4ba1-bf23-020d8573895b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:41.806966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57bd9-81c9-4ba1-bf23-020d8573895b admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:41.807251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57bd9-81c9-4ba1-bf23-020d8573895b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:41.807645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c57bd9-81c9-4ba1-bf23-020d8573895b admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:41.808469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c57bd9-81c9-4ba1-bf23-020d8573895b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:41.817276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c57bd9-81c9-4ba1-bf23-020d8573895b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:41.824056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:41.824800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:41.825150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:41.825498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] PATH_INFO: `/v3/projects/205fc43a29834dd3bdf4265d0e4aa5c2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:41.828497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=205fc43a29834dd3bdf4265d0e4aa5c2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:41.832657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:41.867640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:41.890954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 909/1840] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:41 2022] POST /identity/v3/projects => generated 397 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:41.894197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c57bd9-81c9-4ba1-bf23-020d8573895b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:41.906199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '205fc43a29834dd3bdf4265d0e4aa5c2', 'request_id': 'req-3ebea769-432a-43cb-8526-66164e6e4169'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.913277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '205fc43a29834dd3bdf4265d0e4aa5c2', 'request_id': 'req-3ebea769-432a-43cb-8526-66164e6e4169'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.919871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '205fc43a29834dd3bdf4265d0e4aa5c2', 'request_id': 'req-3ebea769-432a-43cb-8526-66164e6e4169'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.946084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] The token cache is being invalidate because project 205fc43a29834dd3bdf4265d0e4aa5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:41.946607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.947197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3ebea769-432a-43cb-8526-66164e6e4169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:41.949584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 932/1841] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:41 2022] DELETE /identity/v3/projects/205fc43a29834dd3bdf4265d0e4aa5c2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:41.955969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a195-1851-4484-95a0-a4773e903862 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:41.956260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a195-1851-4484-95a0-a4773e903862 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:41.956639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57a195-1851-4484-95a0-a4773e903862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:41.975322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c57bd9-81c9-4ba1-bf23-020d8573895b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:41.980358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:41.980979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:41.981375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:41.981785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:41.982450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:41.984416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:41.986451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-1e57a195-1851-4484-95a0-a4773e903862 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:41.987668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1e57a195-1851-4484-95a0-a4773e903862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:41.996869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 933/1842] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:42.007524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecdc21-0631-4471-97d7-dadab85d2963 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.007800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecdc21-0631-4471-97d7-dadab85d2963 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.008076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecdc21-0631-4471-97d7-dadab85d2963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.018024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 910/1843] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:41 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:42.022390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.024890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:42.039460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a7ecdc21-0631-4471-97d7-dadab85d2963 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:42.040838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a7ecdc21-0631-4471-97d7-dadab85d2963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:42.041550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbcf34a-1eff-49e1-acbc-3e0dc1d284af admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.046928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05743f05-7bbb-48a8-957a-284d258c514c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.047621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05743f05-7bbb-48a8-957a-284d258c514c admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.048029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05743f05-7bbb-48a8-957a-284d258c514c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.048314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05743f05-7bbb-48a8-957a-284d258c514c admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.048968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05743f05-7bbb-48a8-957a-284d258c514c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.050911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05743f05-7bbb-48a8-957a-284d258c514c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.059676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 934/1844] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:42.069065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecdc21-0631-4471-97d7-dadab85d2963 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.080612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 911/1845] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:42.081658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05743f05-7bbb-48a8-957a-284d258c514c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.101969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05743f05-7bbb-48a8-957a-284d258c514c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.102867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecdc21-0631-4471-97d7-dadab85d2963 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.108437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.109377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.109931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.110384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.111560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:42.112120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.114531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.115620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00932d25-d740-4a28-b37b-69a5fd61ff94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.116261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932d25-d740-4a28-b37b-69a5fd61ff94 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.116545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932d25-d740-4a28-b37b-69a5fd61ff94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.116822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00932d25-d740-4a28-b37b-69a5fd61ff94 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.117638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00932d25-d740-4a28-b37b-69a5fd61ff94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.119618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00932d25-d740-4a28-b37b-69a5fd61ff94 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.120199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:42.147993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 912/1846] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:49:42.160073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00932d25-d740-4a28-b37b-69a5fd61ff94 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.163204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 935/1847] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:42.171675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.175231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00932d25-d740-4a28-b37b-69a5fd61ff94 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.180678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a19ff6-bac8-45e6-a819-f1b4eac2001f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.181368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a19ff6-bac8-45e6-a819-f1b4eac2001f admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.181697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a19ff6-bac8-45e6-a819-f1b4eac2001f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.181993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a19ff6-bac8-45e6-a819-f1b4eac2001f admin admin] PATH_INFO: `/v3/projects/0a10dd8c9aaa45bab4ae8ab43e038367/users/d6d399255cc645a3bcd4ef3ac282018a/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.210957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6808b48-bc87-432e-8a87-3eb2859a6a20 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.226010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a19ff6-bac8-45e6-a819-f1b4eac2001f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a10dd8c9aaa45bab4ae8ab43e038367, user_id=d6d399255cc645a3bcd4ef3ac282018a, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.226905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba457f-a22d-45f6-94a6-590ec13f86d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.227609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba457f-a22d-45f6-94a6-590ec13f86d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.227993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba457f-a22d-45f6-94a6-590ec13f86d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.228385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba457f-a22d-45f6-94a6-590ec13f86d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.229164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba457f-a22d-45f6-94a6-590ec13f86d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.231853 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a19ff6-bac8-45e6-a819-f1b4eac2001f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.233969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba457f-a22d-45f6-94a6-590ec13f86d0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.272418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 913/1848] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:42 2022] PUT /identity/v3/projects/0a10dd8c9aaa45bab4ae8ab43e038367/users/d6d399255cc645a3bcd4ef3ac282018a/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:42.282014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf097-ec47-40b5-8919-db4520b623e7 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.282643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf097-ec47-40b5-8919-db4520b623e7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.283153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf097-ec47-40b5-8919-db4520b623e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.303229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 936/1849] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:42 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:42.307963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba457f-a22d-45f6-94a6-590ec13f86d0 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.330994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-13dbf097-ec47-40b5-8919-db4520b623e7 None None] MFA Rules not processed for user `d6d399255cc645a3bcd4ef3ac282018a`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:42.332645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-13dbf097-ec47-40b5-8919-db4520b623e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:42.409969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba457f-a22d-45f6-94a6-590ec13f86d0 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.420978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3401619-cc7a-40f5-a4e7-81638d57aea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.421650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3401619-cc7a-40f5-a4e7-81638d57aea8 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.421938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3401619-cc7a-40f5-a4e7-81638d57aea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.422213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3401619-cc7a-40f5-a4e7-81638d57aea8 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.424029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3401619-cc7a-40f5-a4e7-81638d57aea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.426683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3401619-cc7a-40f5-a4e7-81638d57aea8 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.442736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 914/1850] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:42 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:42.449584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf097-ec47-40b5-8919-db4520b623e7 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.466500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf097-ec47-40b5-8919-db4520b623e7 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.471165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 937/1851] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:42 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:42.475686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3401619-cc7a-40f5-a4e7-81638d57aea8 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.479086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0dc50-72c0-4e14-bbbf-43c73cd8f83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.480692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0dc50-72c0-4e14-bbbf-43c73cd8f83f admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.481042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0dc50-72c0-4e14-bbbf-43c73cd8f83f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.481322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0dc50-72c0-4e14-bbbf-43c73cd8f83f admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.482017 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0dc50-72c0-4e14-bbbf-43c73cd8f83f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.484532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d0dc50-72c0-4e14-bbbf-43c73cd8f83f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.487732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3401619-cc7a-40f5-a4e7-81638d57aea8 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.507384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd71d2-bdb3-4036-92de-9b08219fb44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.508271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd71d2-bdb3-4036-92de-9b08219fb44e admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.509303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd71d2-bdb3-4036-92de-9b08219fb44e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.509648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd71d2-bdb3-4036-92de-9b08219fb44e admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.510477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd71d2-bdb3-4036-92de-9b08219fb44e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.512341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd71d2-bdb3-4036-92de-9b08219fb44e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.528951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 938/1853] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:42.529438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 915/1853] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:42 2022] POST /identity/v3/projects => generated 397 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:42.530238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd71d2-bdb3-4036-92de-9b08219fb44e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.531386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0dc50-72c0-4e14-bbbf-43c73cd8f83f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.543673 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:42.545158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd71d2-bdb3-4036-92de-9b08219fb44e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.641984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcac8b-d70b-4eb3-8c6f-e0d34362ae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.642676 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcac8b-d70b-4eb3-8c6f-e0d34362ae98 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.642960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcac8b-d70b-4eb3-8c6f-e0d34362ae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.643234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcac8b-d70b-4eb3-8c6f-e0d34362ae98 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.643875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcac8b-d70b-4eb3-8c6f-e0d34362ae98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.645914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcac8b-d70b-4eb3-8c6f-e0d34362ae98 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.657146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0dc50-72c0-4e14-bbbf-43c73cd8f83f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.668381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e843-6442-46a2-a2c9-9a85b7803857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.672178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e843-6442-46a2-a2c9-9a85b7803857 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.676740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e843-6442-46a2-a2c9-9a85b7803857 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.677245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e843-6442-46a2-a2c9-9a85b7803857 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.678829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75e843-6442-46a2-a2c9-9a85b7803857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.680341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 939/1854] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:42 2022] POST /identity/v3/users => generated 305 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:42.683372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75e843-6442-46a2-a2c9-9a85b7803857 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.683835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcac8b-d70b-4eb3-8c6f-e0d34362ae98 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.694463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcac8b-d70b-4eb3-8c6f-e0d34362ae98 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.700268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd123bf-6dbe-45a3-873f-960da7a3e590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.700891 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd123bf-6dbe-45a3-873f-960da7a3e590 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.701297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd123bf-6dbe-45a3-873f-960da7a3e590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.701653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd123bf-6dbe-45a3-873f-960da7a3e590 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.702367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd123bf-6dbe-45a3-873f-960da7a3e590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.704375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd123bf-6dbe-45a3-873f-960da7a3e590 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.709880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 916/1855] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:42.711036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e843-6442-46a2-a2c9-9a85b7803857 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.720974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 940/1856] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:42.722017 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd123bf-6dbe-45a3-873f-960da7a3e590 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.745056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e843-6442-46a2-a2c9-9a85b7803857 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.745271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd123bf-6dbe-45a3-873f-960da7a3e590 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.750972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033bf6a-d4ec-451a-a50c-50b162182fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.751693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033bf6a-d4ec-451a-a50c-50b162182fde admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.751971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033bf6a-d4ec-451a-a50c-50b162182fde admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.752301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033bf6a-d4ec-451a-a50c-50b162182fde admin admin] PATH_INFO: `/v3/projects/2fb61dd1468f48f8ae0c94b7f2b6be08/users/ac035ad8de794f3bbb10542238baf33a/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.758139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027af90-a3c6-4d68-be2c-c08e7718fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.758848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027af90-a3c6-4d68-be2c-c08e7718fa6a admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.759138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027af90-a3c6-4d68-be2c-c08e7718fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.759420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027af90-a3c6-4d68-be2c-c08e7718fa6a admin admin] PATH_INFO: `/v3/projects/1820f89a28d8403d886c56a25609cb97/users/2e5246ec28d54146a8dee10dc9a76f84/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.769240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033bf6a-d4ec-451a-a50c-50b162182fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb61dd1468f48f8ae0c94b7f2b6be08, user_id=ac035ad8de794f3bbb10542238baf33a, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.772018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033bf6a-d4ec-451a-a50c-50b162182fde admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.792370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 941/1857] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:42 2022] PUT /identity/v3/projects/2fb61dd1468f48f8ae0c94b7f2b6be08/users/ac035ad8de794f3bbb10542238baf33a/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:42.793374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033bf6a-d4ec-451a-a50c-50b162182fde admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:42.803172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027af90-a3c6-4d68-be2c-c08e7718fa6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1820f89a28d8403d886c56a25609cb97, user_id=2e5246ec28d54146a8dee10dc9a76f84, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.804336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:42.806438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027af90-a3c6-4d68-be2c-c08e7718fa6a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.850560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 917/1858] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:42 2022] PUT /identity/v3/projects/1820f89a28d8403d886c56a25609cb97/users/2e5246ec28d54146a8dee10dc9a76f84/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:42.852244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac079e-e1e7-4892-97d5-0b3a37e45069 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.852538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac079e-e1e7-4892-97d5-0b3a37e45069 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.852818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac079e-e1e7-4892-97d5-0b3a37e45069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.880111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033bf6a-d4ec-451a-a50c-50b162182fde admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:42.929476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-fcac079e-e1e7-4892-97d5-0b3a37e45069 None None] MFA Rules not processed for user `2e5246ec28d54146a8dee10dc9a76f84`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:42.934440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fcac079e-e1e7-4892-97d5-0b3a37e45069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:42.965359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290d9b5-18ae-4a6a-bb5f-9ef5359d48bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:42.966293 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290d9b5-18ae-4a6a-bb5f-9ef5359d48bb admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:42.966577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290d9b5-18ae-4a6a-bb5f-9ef5359d48bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:42.966851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290d9b5-18ae-4a6a-bb5f-9ef5359d48bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:42.967511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290d9b5-18ae-4a6a-bb5f-9ef5359d48bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:42.969200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290d9b5-18ae-4a6a-bb5f-9ef5359d48bb admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:42.982326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 942/1859] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:49:42.986649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290d9b5-18ae-4a6a-bb5f-9ef5359d48bb admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.014900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290d9b5-18ae-4a6a-bb5f-9ef5359d48bb admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.031196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabeabb-0521-463b-8c63-2c446a20d168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.034074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabeabb-0521-463b-8c63-2c446a20d168 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.034226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabeabb-0521-463b-8c63-2c446a20d168 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.034404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabeabb-0521-463b-8c63-2c446a20d168 admin admin] PATH_INFO: `/v3/projects/2fb61dd1468f48f8ae0c94b7f2b6be08/users/ac035ad8de794f3bbb10542238baf33a/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.039998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabeabb-0521-463b-8c63-2c446a20d168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb61dd1468f48f8ae0c94b7f2b6be08, user_id=ac035ad8de794f3bbb10542238baf33a, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.046652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabeabb-0521-463b-8c63-2c446a20d168 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.088125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 943/1860] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:42 2022] PUT /identity/v3/projects/2fb61dd1468f48f8ae0c94b7f2b6be08/users/ac035ad8de794f3bbb10542238baf33a/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:43.093123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6c7b-f122-43cf-aa56-8f39b3ee3fec None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.093458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6c7b-f122-43cf-aa56-8f39b3ee3fec None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.093702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6c7b-f122-43cf-aa56-8f39b3ee3fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.108743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 918/1861] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:43.110281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac079e-e1e7-4892-97d5-0b3a37e45069 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.130892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac079e-e1e7-4892-97d5-0b3a37e45069 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.146950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-340d6c7b-f122-43cf-aa56-8f39b3ee3fec None None] MFA Rules not processed for user `ac035ad8de794f3bbb10542238baf33a`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:43.149464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-340d6c7b-f122-43cf-aa56-8f39b3ee3fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:43.235749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd4fec-6b9f-44b1-af53-5758c5cf9cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.238066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd4fec-6b9f-44b1-af53-5758c5cf9cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.238597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd4fec-6b9f-44b1-af53-5758c5cf9cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.239109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd4fec-6b9f-44b1-af53-5758c5cf9cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.240689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd4fec-6b9f-44b1-af53-5758c5cf9cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.243046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd4fec-6b9f-44b1-af53-5758c5cf9cc1 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.249834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 944/1862] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:43 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:43.262684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6c7b-f122-43cf-aa56-8f39b3ee3fec None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.264209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-340d6c7b-f122-43cf-aa56-8f39b3ee3fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:43.272152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:43.289954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 919/1863] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:43 2022] POST /identity/v3/projects => generated 389 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:43.297445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd4fec-6b9f-44b1-af53-5758c5cf9cc1 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.313495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:43.361740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6c7b-f122-43cf-aa56-8f39b3ee3fec None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.402514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd4fec-6b9f-44b1-af53-5758c5cf9cc1 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.408517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5cf10-e6cb-491a-96ce-57b21d93207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.409305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5cf10-e6cb-491a-96ce-57b21d93207f admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.409728 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5cf10-e6cb-491a-96ce-57b21d93207f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.410110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5cf10-e6cb-491a-96ce-57b21d93207f admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.411005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5cf10-e6cb-491a-96ce-57b21d93207f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.413243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5cf10-e6cb-491a-96ce-57b21d93207f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.441160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ad3b7-e3df-4c8c-b795-84c85715a23c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.444318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 920/1864] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:43 2022] POST /identity/v3/users => generated 301 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:43.445082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ad3b7-e3df-4c8c-b795-84c85715a23c service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.445402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ad3b7-e3df-4c8c-b795-84c85715a23c service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.445727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ad3b7-e3df-4c8c-b795-84c85715a23c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.446977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5cf10-e6cb-491a-96ce-57b21d93207f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.461967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5cf10-e6cb-491a-96ce-57b21d93207f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.470090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdc641-4732-45a6-8f63-ad6ea781b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.471438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdc641-4732-45a6-8f63-ad6ea781b931 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.472080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdc641-4732-45a6-8f63-ad6ea781b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.472645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bdc641-4732-45a6-8f63-ad6ea781b931 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.473766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdc641-4732-45a6-8f63-ad6ea781b931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.476484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bdc641-4732-45a6-8f63-ad6ea781b931 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.491595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 921/1865] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:43.495884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdc641-4732-45a6-8f63-ad6ea781b931 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.510961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bdc641-4732-45a6-8f63-ad6ea781b931 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.526997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb501a9-c9b9-4662-8aa1-e6f4c5d794b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.528973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb501a9-c9b9-4662-8aa1-e6f4c5d794b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.529798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb501a9-c9b9-4662-8aa1-e6f4c5d794b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.530390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb501a9-c9b9-4662-8aa1-e6f4c5d794b9 admin admin] PATH_INFO: `/v3/projects/de876a43cacc4750a1492ae0b3ef6a9c/users/d6d06afba9874e278b1250fe6eb5a0c2/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.562774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ad3b7-e3df-4c8c-b795-84c85715a23c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.563339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb501a9-c9b9-4662-8aa1-e6f4c5d794b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de876a43cacc4750a1492ae0b3ef6a9c, user_id=d6d06afba9874e278b1250fe6eb5a0c2, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.564751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ad3b7-e3df-4c8c-b795-84c85715a23c service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.567562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb501a9-c9b9-4662-8aa1-e6f4c5d794b9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.598754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 922/1866] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:43 2022] PUT /identity/v3/projects/de876a43cacc4750a1492ae0b3ef6a9c/users/d6d06afba9874e278b1250fe6eb5a0c2/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:43.598926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 945/1867] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:43 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:43.600786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb501a9-c9b9-4662-8aa1-e6f4c5d794b9 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.708349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb501a9-c9b9-4662-8aa1-e6f4c5d794b9 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.711349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ad3b7-e3df-4c8c-b795-84c85715a23c service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.713917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6744-44f3-4aa7-a77b-e0c92ba34dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.714541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6744-44f3-4aa7-a77b-e0c92ba34dfb admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.714821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6744-44f3-4aa7-a77b-e0c92ba34dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.715098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6744-44f3-4aa7-a77b-e0c92ba34dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.715750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd6744-44f3-4aa7-a77b-e0c92ba34dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.717467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd6744-44f3-4aa7-a77b-e0c92ba34dfb admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.721881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ad3b7-e3df-4c8c-b795-84c85715a23c service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.730562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 923/1868] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:43.734537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6744-44f3-4aa7-a77b-e0c92ba34dfb admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.751030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6744-44f3-4aa7-a77b-e0c92ba34dfb admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:43.757210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b176764-f601-4baf-9b3a-890fcd11d3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.757914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b176764-f601-4baf-9b3a-890fcd11d3fb admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.758260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b176764-f601-4baf-9b3a-890fcd11d3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.758471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b176764-f601-4baf-9b3a-890fcd11d3fb admin admin] PATH_INFO: `/v3/projects/de876a43cacc4750a1492ae0b3ef6a9c/users/d6d06afba9874e278b1250fe6eb5a0c2/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.761355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b176764-f601-4baf-9b3a-890fcd11d3fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=de876a43cacc4750a1492ae0b3ef6a9c, user_id=d6d06afba9874e278b1250fe6eb5a0c2, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.765108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b176764-f601-4baf-9b3a-890fcd11d3fb admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.778148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 924/1869] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:43 2022] PUT /identity/v3/projects/de876a43cacc4750a1492ae0b3ef6a9c/users/d6d06afba9874e278b1250fe6eb5a0c2/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:43.788193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:43.788523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dffba7-6fdb-4046-9990-7cf30fc48f4f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.788813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dffba7-6fdb-4046-9990-7cf30fc48f4f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.788933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:43.789108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:43.789205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dffba7-6fdb-4046-9990-7cf30fc48f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.789413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:43.828321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e3dffba7-6fdb-4046-9990-7cf30fc48f4f None None] MFA Rules not processed for user `d6d06afba9874e278b1250fe6eb5a0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:43.830296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e3dffba7-6fdb-4046-9990-7cf30fc48f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:43.870283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:43.880071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:43.909381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:43.910947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:43.938483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 946/1870] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:43 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:43.982200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 925/1871] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:43 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:43.996328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:43.997901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:44.013707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52828c3-374a-473a-9c72-94d42991c4b2 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:44.096205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db1f55-703d-4888-9064-441ac5ad9ceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:44.097347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db1f55-703d-4888-9064-441ac5ad9ceb service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:44.097896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db1f55-703d-4888-9064-441ac5ad9ceb service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:44.098556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db1f55-703d-4888-9064-441ac5ad9ceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:44.119987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db1f55-703d-4888-9064-441ac5ad9ceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:44.123089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db1f55-703d-4888-9064-441ac5ad9ceb service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:44.143858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 947/1872] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:44.357437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dffba7-6fdb-4046-9990-7cf30fc48f4f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:44.524068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dffba7-6fdb-4046-9990-7cf30fc48f4f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:44.536137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:44.538148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:44.538444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:44.539024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] PATH_INFO: `/v3/users/2e5246ec28d54146a8dee10dc9a76f84` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:44.546102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5246ec28d54146a8dee10dc9a76f84)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:44.548656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:44.608907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5246ec28d54146a8dee10dc9a76f84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:44.619704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5246ec28d54146a8dee10dc9a76f84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:44.621565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5246ec28d54146a8dee10dc9a76f84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:44.643682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5246ec28d54146a8dee10dc9a76f84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:44.653318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5246ec28d54146a8dee10dc9a76f84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:44.660055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 926/1873] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:44 2022] DELETE /identity/v3/users/2e5246ec28d54146a8dee10dc9a76f84 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:44.664362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7b428-26f6-4812-8038-54d72d779485 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:44.664845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7b428-26f6-4812-8038-54d72d779485 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:44.664922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7b428-26f6-4812-8038-54d72d779485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:44.684832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-c1c7b428-26f6-4812-8038-54d72d779485 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:44.686203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c1c7b428-26f6-4812-8038-54d72d779485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:44.762476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 948/1874] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:44.773656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:44.775369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:44.857284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:44.892628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092165f5-c355-4e47-9181-378ef44b9f22 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:44.899536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afede0-fee2-4d08-8b2f-c06caf81c330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:44.900241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afede0-fee2-4d08-8b2f-c06caf81c330 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:44.900598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afede0-fee2-4d08-8b2f-c06caf81c330 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:44.900970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afede0-fee2-4d08-8b2f-c06caf81c330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:44.913840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afede0-fee2-4d08-8b2f-c06caf81c330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:44.916094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afede0-fee2-4d08-8b2f-c06caf81c330 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:44.933443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 927/1875] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:45.271765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7b428-26f6-4812-8038-54d72d779485 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:45.273568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c1c7b428-26f6-4812-8038-54d72d779485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:45.290339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7b428-26f6-4812-8038-54d72d779485 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:45.295240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:45.295873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:45.296153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:45.296425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] PATH_INFO: `/v3/projects/1820f89a28d8403d886c56a25609cb97` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:45.297573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1820f89a28d8403d886c56a25609cb97)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:45.299800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:45.361716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1820f89a28d8403d886c56a25609cb97', 'request_id': 'req-73a6557b-3638-4099-a34a-723118433a4b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.370609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1820f89a28d8403d886c56a25609cb97', 'request_id': 'req-73a6557b-3638-4099-a34a-723118433a4b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.382157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1820f89a28d8403d886c56a25609cb97', 'request_id': 'req-73a6557b-3638-4099-a34a-723118433a4b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.397373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] The token cache is being invalidate because project 1820f89a28d8403d886c56a25609cb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:45.397717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.398480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.400113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 949/1876] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:45 2022] DELETE /identity/v3/projects/1820f89a28d8403d886c56a25609cb97 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:45.405041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afede0-fee2-4d08-8b2f-c06caf81c330 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:45.416640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afede0-fee2-4d08-8b2f-c06caf81c330 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:45.480973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:45.481647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:45.481920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:45.482198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] PATH_INFO: `/v3/users/d6d06afba9874e278b1250fe6eb5a0c2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:45.483311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d06afba9874e278b1250fe6eb5a0c2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:45.485264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:45.554418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d06afba9874e278b1250fe6eb5a0c2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.566019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d06afba9874e278b1250fe6eb5a0c2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.567125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d06afba9874e278b1250fe6eb5a0c2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.584137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d06afba9874e278b1250fe6eb5a0c2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.595530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a901a285-d9c1-4b87-969b-d8962016dfd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d06afba9874e278b1250fe6eb5a0c2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:45.599196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 928/1877] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:45 2022] DELETE /identity/v3/users/d6d06afba9874e278b1250fe6eb5a0c2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:45.981392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:45.983075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:45.999832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6557b-3638-4099-a34a-723118433a4b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:46.013766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:46.088013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:46.088680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.088962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.089237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] PATH_INFO: `/v3/projects/de876a43cacc4750a1492ae0b3ef6a9c` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.090450 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de876a43cacc4750a1492ae0b3ef6a9c)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:46.092546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:46.167824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de876a43cacc4750a1492ae0b3ef6a9c', 'request_id': 'req-2152ec09-1ad8-4d2a-9786-9b2fda175427'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:46.185746 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de876a43cacc4750a1492ae0b3ef6a9c', 'request_id': 'req-2152ec09-1ad8-4d2a-9786-9b2fda175427'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:46.193843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de876a43cacc4750a1492ae0b3ef6a9c', 'request_id': 'req-2152ec09-1ad8-4d2a-9786-9b2fda175427'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:46.210088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] The token cache is being invalidate because project de876a43cacc4750a1492ae0b3ef6a9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:46.210491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:46.211367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2152ec09-1ad8-4d2a-9786-9b2fda175427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:46.214228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 950/1878] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:45 2022] DELETE /identity/v3/projects/de876a43cacc4750a1492ae0b3ef6a9c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:46.218882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b6b94-b3f6-42c1-9385-e759ee280dc2 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.219165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b6b94-b3f6-42c1-9385-e759ee280dc2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.219446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b6b94-b3f6-42c1-9385-e759ee280dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.243375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8d6b6b94-b3f6-42c1-9385-e759ee280dc2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:46.245286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8d6b6b94-b3f6-42c1-9385-e759ee280dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:46.347171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:46.371573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 929/1879] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:46.380767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a644a6-a760-4fca-a0ac-23d25be352e2 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.381049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a644a6-a760-4fca-a0ac-23d25be352e2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.381326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a644a6-a760-4fca-a0ac-23d25be352e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.430794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d9a644a6-a760-4fca-a0ac-23d25be352e2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:46.434158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d9a644a6-a760-4fca-a0ac-23d25be352e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:46.452982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 951/1880] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:46.459411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b6b94-b3f6-42c1-9385-e759ee280dc2 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:46.474043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b6b94-b3f6-42c1-9385-e759ee280dc2 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:46.492651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:46.493894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.493894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.494323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.495256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:46.495791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:46.503073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:46.536344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 930/1881] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:46.542301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a644a6-a760-4fca-a0ac-23d25be352e2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:46.555396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a644a6-a760-4fca-a0ac-23d25be352e2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:46.562095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a94d15-825f-4e22-b920-550a201715ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:46.574758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a94d15-825f-4e22-b920-550a201715ca admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.575061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a94d15-825f-4e22-b920-550a201715ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.575342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a94d15-825f-4e22-b920-550a201715ca admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.576019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a94d15-825f-4e22-b920-550a201715ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:46.580716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a94d15-825f-4e22-b920-550a201715ca admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:46.617488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 952/1882] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:46 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:46.622283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:46.635999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea16f2-9cfc-49aa-894c-5ce71727d01f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:46.747946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bb4c0-4acf-483d-87c8-a7be0a6bddde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:46.749034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bb4c0-4acf-483d-87c8-a7be0a6bddde admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.749751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bb4c0-4acf-483d-87c8-a7be0a6bddde admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.750209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bb4c0-4acf-483d-87c8-a7be0a6bddde admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.751588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bb4c0-4acf-483d-87c8-a7be0a6bddde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:46.755864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bb4c0-4acf-483d-87c8-a7be0a6bddde admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:46.789420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 931/1883] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:46 2022] POST /identity/v3/users => generated 284 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:46.797117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a94d15-825f-4e22-b920-550a201715ca admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:46.812475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a94d15-825f-4e22-b920-550a201715ca admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:46.819355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6974aa-5da2-4a52-8f59-79ec46356d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:46.820070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6974aa-5da2-4a52-8f59-79ec46356d6a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.820713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6974aa-5da2-4a52-8f59-79ec46356d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.821300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6974aa-5da2-4a52-8f59-79ec46356d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.822450 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6974aa-5da2-4a52-8f59-79ec46356d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:46.824452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6974aa-5da2-4a52-8f59-79ec46356d6a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:46.841440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 953/1884] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:46.842898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bb4c0-4acf-483d-87c8-a7be0a6bddde admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:46.854955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bb4c0-4acf-483d-87c8-a7be0a6bddde admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:46.860229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0d67c-91b2-42fc-ad7b-e6eeecbe50c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:46.861041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d67c-91b2-42fc-ad7b-e6eeecbe50c0 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.861384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d67c-91b2-42fc-ad7b-e6eeecbe50c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.861735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d67c-91b2-42fc-ad7b-e6eeecbe50c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.862387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0d67c-91b2-42fc-ad7b-e6eeecbe50c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:46.864181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0d67c-91b2-42fc-ad7b-e6eeecbe50c0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:46.877145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 932/1885] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:46.881502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6974aa-5da2-4a52-8f59-79ec46356d6a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:46.894373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6974aa-5da2-4a52-8f59-79ec46356d6a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:46.900096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7822239-7c28-47db-881c-634efbadc2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:46.901364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7822239-7c28-47db-881c-634efbadc2ea admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.901364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7822239-7c28-47db-881c-634efbadc2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.901592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7822239-7c28-47db-881c-634efbadc2ea admin admin] PATH_INFO: `/v3/projects/b292eea3b29848f39946afdcd7c3b905/users/b76231620dbf4ff686b5376318c12949/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.918997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7822239-7c28-47db-881c-634efbadc2ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b292eea3b29848f39946afdcd7c3b905, user_id=b76231620dbf4ff686b5376318c12949, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:46.921875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7822239-7c28-47db-881c-634efbadc2ea admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:46.937929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 954/1886] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:46 2022] PUT /identity/v3/projects/b292eea3b29848f39946afdcd7c3b905/users/b76231620dbf4ff686b5376318c12949/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:46.948721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776937e0-c900-46cc-ba14-058164fd4bcc None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:46.949103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776937e0-c900-46cc-ba14-058164fd4bcc None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:46.949428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776937e0-c900-46cc-ba14-058164fd4bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:46.984026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-776937e0-c900-46cc-ba14-058164fd4bcc None None] MFA Rules not processed for user `b76231620dbf4ff686b5376318c12949`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:46.985489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-776937e0-c900-46cc-ba14-058164fd4bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:47.086085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 933/1887] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:47.090486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7822239-7c28-47db-881c-634efbadc2ea admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:47.135010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:47.160497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7822239-7c28-47db-881c-634efbadc2ea admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:47.165162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc1472-713a-477b-af0a-ab36dc5a8130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:47.165797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc1472-713a-477b-af0a-ab36dc5a8130 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.166178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc1472-713a-477b-af0a-ab36dc5a8130 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.166500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cc1472-713a-477b-af0a-ab36dc5a8130 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.167003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc1472-713a-477b-af0a-ab36dc5a8130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:47.168900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cc1472-713a-477b-af0a-ab36dc5a8130 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:47.198026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 955/1888] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:47 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:47.202390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776937e0-c900-46cc-ba14-058164fd4bcc None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:47.213432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776937e0-c900-46cc-ba14-058164fd4bcc None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:47.265944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:47.278337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6f84d-e7a0-4c14-909b-7a6269a4eb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:47.279367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6f84d-e7a0-4c14-909b-7a6269a4eb0e admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.279367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6f84d-e7a0-4c14-909b-7a6269a4eb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.279525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6f84d-e7a0-4c14-909b-7a6269a4eb0e admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.280248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6f84d-e7a0-4c14-909b-7a6269a4eb0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:47.282281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6f84d-e7a0-4c14-909b-7a6269a4eb0e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:47.308392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 934/1889] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:47 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:47.312871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc1472-713a-477b-af0a-ab36dc5a8130 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:47.330064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cc1472-713a-477b-af0a-ab36dc5a8130 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:47.335527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836b03-42f7-4b85-94e5-8ea947210224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:47.336170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836b03-42f7-4b85-94e5-8ea947210224 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.336449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836b03-42f7-4b85-94e5-8ea947210224 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.336718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9836b03-42f7-4b85-94e5-8ea947210224 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.337361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9836b03-42f7-4b85-94e5-8ea947210224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:47.339044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9836b03-42f7-4b85-94e5-8ea947210224 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:47.351464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 956/1890] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:47.355834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6f84d-e7a0-4c14-909b-7a6269a4eb0e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:47.365969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6f84d-e7a0-4c14-909b-7a6269a4eb0e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:47.370991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d703fa-3f13-42a2-b814-97c638992ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:47.371689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d703fa-3f13-42a2-b814-97c638992ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.371982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d703fa-3f13-42a2-b814-97c638992ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.372277 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d703fa-3f13-42a2-b814-97c638992ca5 admin admin] PATH_INFO: `/v3/projects/89cbd3e76f364dd995fe9bbe6720600c/users/21ad3885e4bb4ec3ae5b6c13598e813c/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.388855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d703fa-3f13-42a2-b814-97c638992ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cbd3e76f364dd995fe9bbe6720600c, user_id=21ad3885e4bb4ec3ae5b6c13598e813c, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:47.391702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d703fa-3f13-42a2-b814-97c638992ca5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:47.404671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 935/1891] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:47 2022] PUT /identity/v3/projects/89cbd3e76f364dd995fe9bbe6720600c/users/21ad3885e4bb4ec3ae5b6c13598e813c/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:47.408834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836b03-42f7-4b85-94e5-8ea947210224 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:47.418624 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9836b03-42f7-4b85-94e5-8ea947210224 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:47.480263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d966e60a-e9ac-4b49-8626-492723a6719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:47.481260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966e60a-e9ac-4b49-8626-492723a6719f admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.481486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966e60a-e9ac-4b49-8626-492723a6719f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.481886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d966e60a-e9ac-4b49-8626-492723a6719f admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.482634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966e60a-e9ac-4b49-8626-492723a6719f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:47.484578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d966e60a-e9ac-4b49-8626-492723a6719f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:47.494801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 957/1892] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:47.499528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d703fa-3f13-42a2-b814-97c638992ca5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:47.513855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d703fa-3f13-42a2-b814-97c638992ca5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:47.519597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:47.520274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.520538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.520864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] PATH_INFO: `/v3/projects/89cbd3e76f364dd995fe9bbe6720600c/users/21ad3885e4bb4ec3ae5b6c13598e813c/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.523477 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=89cbd3e76f364dd995fe9bbe6720600c, user_id=21ad3885e4bb4ec3ae5b6c13598e813c, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:47.527588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:47.546259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 936/1893] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:47 2022] PUT /identity/v3/projects/89cbd3e76f364dd995fe9bbe6720600c/users/21ad3885e4bb4ec3ae5b6c13598e813c/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:47.564544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89910463-0126-4c02-b6a4-aad63aa7a617 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.564821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89910463-0126-4c02-b6a4-aad63aa7a617 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.565068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89910463-0126-4c02-b6a4-aad63aa7a617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.595986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-89910463-0126-4c02-b6a4-aad63aa7a617 None None] MFA Rules not processed for user `21ad3885e4bb4ec3ae5b6c13598e813c`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:47.597126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-89910463-0126-4c02-b6a4-aad63aa7a617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:47.674955 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 958/1894] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:47.687297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:47.689029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:47.747755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b91c3-b5ec-4241-be0c-4d8dbb71e6ca admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:47.751436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:47.752048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.752346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.752619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.761745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:47.763194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:47.775119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 937/1895] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:47.910100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89910463-0126-4c02-b6a4-aad63aa7a617 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:47.921080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89910463-0126-4c02-b6a4-aad63aa7a617 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:47.924938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d9123-47df-4613-8712-3337da9a3639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:47.925547 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d9123-47df-4613-8712-3337da9a3639 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:47.925901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d9123-47df-4613-8712-3337da9a3639 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:47.926175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d9123-47df-4613-8712-3337da9a3639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:47.965127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:48.002225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d9123-47df-4613-8712-3337da9a3639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:48.007718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d9123-47df-4613-8712-3337da9a3639 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:48.023269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 959/1896] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:48.618106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:48.619596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:48.702491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c5858-dba6-43fa-ad90-64d646210c67 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:48.707622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:48.708802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:48.709095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:48.709369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] PATH_INFO: `/v3/users/d6d399255cc645a3bcd4ef3ac282018a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:48.710538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6d399255cc645a3bcd4ef3ac282018a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:48.712478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:48.774532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6d399255cc645a3bcd4ef3ac282018a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:48.778254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:48.784321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6d399255cc645a3bcd4ef3ac282018a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:48.785251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6d399255cc645a3bcd4ef3ac282018a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:48.813843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6d399255cc645a3bcd4ef3ac282018a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:48.819179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6d399255cc645a3bcd4ef3ac282018a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:48.832989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 938/1897] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:48 2022] DELETE /identity/v3/users/d6d399255cc645a3bcd4ef3ac282018a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:48.837723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91477-1a65-476f-92de-45e7b17cf5c0 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:48.838774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91477-1a65-476f-92de-45e7b17cf5c0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:48.839883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c91477-1a65-476f-92de-45e7b17cf5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:48.862391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-07c91477-1a65-476f-92de-45e7b17cf5c0 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:48.863647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-07c91477-1a65-476f-92de-45e7b17cf5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:48.931588 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 960/1898] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:48.944379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:48.945937 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:49.030354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada602b-a6d1-4095-82c3-7214f58d8a5b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:49.037585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c6e7-af44-4e29-90ac-0d8a530dbb9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:49.038472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c6e7-af44-4e29-90ac-0d8a530dbb9c service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:49.039067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c6e7-af44-4e29-90ac-0d8a530dbb9c service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:49.039351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c6e7-af44-4e29-90ac-0d8a530dbb9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:49.050881 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1c6e7-af44-4e29-90ac-0d8a530dbb9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:49.052385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1c6e7-af44-4e29-90ac-0d8a530dbb9c service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:49.068899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 939/1899] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:49.357786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c91477-1a65-476f-92de-45e7b17cf5c0 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:49.359459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-07c91477-1a65-476f-92de-45e7b17cf5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:49.390200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c91477-1a65-476f-92de-45e7b17cf5c0 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:49.395206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:49.396519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:49.396799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:49.397073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] PATH_INFO: `/v3/projects/0a10dd8c9aaa45bab4ae8ab43e038367` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:49.398350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a10dd8c9aaa45bab4ae8ab43e038367)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:49.400309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:49.456911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a10dd8c9aaa45bab4ae8ab43e038367', 'request_id': 'req-eb301a5b-504f-428a-a574-7631dc3e32a9'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.464928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a10dd8c9aaa45bab4ae8ab43e038367', 'request_id': 'req-eb301a5b-504f-428a-a574-7631dc3e32a9'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.472769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a10dd8c9aaa45bab4ae8ab43e038367', 'request_id': 'req-eb301a5b-504f-428a-a574-7631dc3e32a9'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.493267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] The token cache is being invalidate because project 0a10dd8c9aaa45bab4ae8ab43e038367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:49.494844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.497763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.499645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 961/1900] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:49 2022] DELETE /identity/v3/projects/0a10dd8c9aaa45bab4ae8ab43e038367 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:49.503555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c6e7-af44-4e29-90ac-0d8a530dbb9c service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:49.514299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1c6e7-af44-4e29-90ac-0d8a530dbb9c service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:49.580501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:49.581349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:49.581662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:49.581946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] PATH_INFO: `/v3/users/ac035ad8de794f3bbb10542238baf33a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:49.583091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac035ad8de794f3bbb10542238baf33a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:49.585050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:49.649984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac035ad8de794f3bbb10542238baf33a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.666254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac035ad8de794f3bbb10542238baf33a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.666935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac035ad8de794f3bbb10542238baf33a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.688100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac035ad8de794f3bbb10542238baf33a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.689495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:49.702464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-618e6153-a0b5-44da-b50b-c40fb61525de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac035ad8de794f3bbb10542238baf33a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:49.723000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 940/1901] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:49 2022] DELETE /identity/v3/users/ac035ad8de794f3bbb10542238baf33a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:50.115906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:50.118036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:50.136989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb301a5b-504f-428a-a574-7631dc3e32a9 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:50.222231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:50.236721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:50.237641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.237641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.237933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] PATH_INFO: `/v3/projects/2fb61dd1468f48f8ae0c94b7f2b6be08` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.239476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb61dd1468f48f8ae0c94b7f2b6be08)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:50.241791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:50.304230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb61dd1468f48f8ae0c94b7f2b6be08', 'request_id': 'req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:50.311500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb61dd1468f48f8ae0c94b7f2b6be08', 'request_id': 'req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:50.326621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb61dd1468f48f8ae0c94b7f2b6be08', 'request_id': 'req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:50.340841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] The token cache is being invalidate because project 2fb61dd1468f48f8ae0c94b7f2b6be08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:50.342050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:50.342695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4cbf6d7e-571a-47ed-986f-a30b7b0f8fb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:50.345450 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 962/1902] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:50 2022] DELETE /identity/v3/projects/2fb61dd1468f48f8ae0c94b7f2b6be08 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:50.350738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2efd6-6a53-4542-828b-69839a79f9cc None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.351028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2efd6-6a53-4542-828b-69839a79f9cc None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.351312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2efd6-6a53-4542-828b-69839a79f9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.378830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-67d2efd6-6a53-4542-828b-69839a79f9cc None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:50.380841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-67d2efd6-6a53-4542-828b-69839a79f9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:50.462893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 941/1903] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:50.471399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a002392-ea2c-46d9-a1cf-ea3ed04d4678 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.471682 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a002392-ea2c-46d9-a1cf-ea3ed04d4678 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.471955 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a002392-ea2c-46d9-a1cf-ea3ed04d4678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.502029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-2a002392-ea2c-46d9-a1cf-ea3ed04d4678 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:50.504078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2a002392-ea2c-46d9-a1cf-ea3ed04d4678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:50.513708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 963/1904] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:50.518280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2efd6-6a53-4542-828b-69839a79f9cc None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:50.528641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2efd6-6a53-4542-828b-69839a79f9cc None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:50.546337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:50.546976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.547180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.547570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.548086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:50.548398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:50.550443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:50.568032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 942/1905] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:50.572168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a002392-ea2c-46d9-a1cf-ea3ed04d4678 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:50.582018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a002392-ea2c-46d9-a1cf-ea3ed04d4678 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:50.587382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa9fa6-0737-4978-a4de-3b29271c3d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:50.587382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa9fa6-0737-4978-a4de-3b29271c3d31 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.587775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa9fa6-0737-4978-a4de-3b29271c3d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.587937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa9fa6-0737-4978-a4de-3b29271c3d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.588605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa9fa6-0737-4978-a4de-3b29271c3d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:50.590612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaa9fa6-0737-4978-a4de-3b29271c3d31 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:50.625281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 964/1906] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:50 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:50.626857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:50.639072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6b052-b123-4160-b140-d8498fb0dde4 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:50.706004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771b9cf-c737-48fb-b039-6d13c6266734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:50.706785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771b9cf-c737-48fb-b039-6d13c6266734 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.707094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771b9cf-c737-48fb-b039-6d13c6266734 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.707441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7771b9cf-c737-48fb-b039-6d13c6266734 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.709646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771b9cf-c737-48fb-b039-6d13c6266734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:50.711630 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7771b9cf-c737-48fb-b039-6d13c6266734 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:50.737505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 943/1907] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:50 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:50.742129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa9fa6-0737-4978-a4de-3b29271c3d31 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:50.753569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaa9fa6-0737-4978-a4de-3b29271c3d31 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:50.766653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568659d-4d64-4032-8503-bf4817be7d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:50.767337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568659d-4d64-4032-8503-bf4817be7d47 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.767684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568659d-4d64-4032-8503-bf4817be7d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.767959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568659d-4d64-4032-8503-bf4817be7d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.768623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568659d-4d64-4032-8503-bf4817be7d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:50.770493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568659d-4d64-4032-8503-bf4817be7d47 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:50.786667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 965/1908] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:50.790578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771b9cf-c737-48fb-b039-6d13c6266734 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:50.805139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7771b9cf-c737-48fb-b039-6d13c6266734 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:50.821339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e096594e-e2bf-48ed-a1cb-096adaed5b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:50.822359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096594e-e2bf-48ed-a1cb-096adaed5b0d admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.822851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096594e-e2bf-48ed-a1cb-096adaed5b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.823228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e096594e-e2bf-48ed-a1cb-096adaed5b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.823967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096594e-e2bf-48ed-a1cb-096adaed5b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:50.825840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e096594e-e2bf-48ed-a1cb-096adaed5b0d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:50.844858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 944/1909] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:50.850137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568659d-4d64-4032-8503-bf4817be7d47 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:50.861027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568659d-4d64-4032-8503-bf4817be7d47 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:50.882195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:50.882775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.883038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.883449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] PATH_INFO: `/v3/projects/3bcc2144ca4e4ebd981e8e2c0ab073e2/users/5d38236394a64b248341fb948b3a5d92/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.905176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bcc2144ca4e4ebd981e8e2c0ab073e2, user_id=5d38236394a64b248341fb948b3a5d92, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:50.908438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:50.925563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 966/1910] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:50 2022] PUT /identity/v3/projects/3bcc2144ca4e4ebd981e8e2c0ab073e2/users/5d38236394a64b248341fb948b3a5d92/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:50.937380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159da5fa-5b5d-48b5-a52d-092464a3f92e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:50.937831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159da5fa-5b5d-48b5-a52d-092464a3f92e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:50.938210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159da5fa-5b5d-48b5-a52d-092464a3f92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:50.978075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-159da5fa-5b5d-48b5-a52d-092464a3f92e None None] MFA Rules not processed for user `5d38236394a64b248341fb948b3a5d92`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:50.979467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-159da5fa-5b5d-48b5-a52d-092464a3f92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:51.073072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:51.116509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 945/1911] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:51.129755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:51.133001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:51.253538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2db81-795b-4b2c-9e59-1403a8875333 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:51.261028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9dc317-ac95-44cc-843e-41469fd23e5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:51.261701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dc317-ac95-44cc-843e-41469fd23e5f service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:51.261985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dc317-ac95-44cc-843e-41469fd23e5f service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:51.262235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9dc317-ac95-44cc-843e-41469fd23e5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:51.272027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9dc317-ac95-44cc-843e-41469fd23e5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:51.273496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9dc317-ac95-44cc-843e-41469fd23e5f service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:51.283967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 967/1912] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:51.322449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159da5fa-5b5d-48b5-a52d-092464a3f92e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:51.438192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159da5fa-5b5d-48b5-a52d-092464a3f92e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:51.447537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:51.448199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:51.448480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:51.448751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] PATH_INFO: `/v3/users/5d38236394a64b248341fb948b3a5d92` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:51.449968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d38236394a64b248341fb948b3a5d92)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:51.452008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:51.516981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d38236394a64b248341fb948b3a5d92'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:51.525195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d38236394a64b248341fb948b3a5d92'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:51.526027 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d38236394a64b248341fb948b3a5d92'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:51.551645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d38236394a64b248341fb948b3a5d92'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:51.564931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d38236394a64b248341fb948b3a5d92'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:51.575255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 946/1913] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:51 2022] DELETE /identity/v3/users/5d38236394a64b248341fb948b3a5d92 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:51.587472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf79c3-b940-4004-9c86-282db52e8cbe None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:51.587803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf79c3-b940-4004-9c86-282db52e8cbe None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:51.588160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf79c3-b940-4004-9c86-282db52e8cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:51.619775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3bcf79c3-b940-4004-9c86-282db52e8cbe None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:51.621064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3bcf79c3-b940-4004-9c86-282db52e8cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:51.640629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:51.700292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 968/1914] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:51.727519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:51.732216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:51.822750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac399eb-c4f5-49b2-9dc0-a45da78fa930 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:51.827181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9d3f-b48b-466f-a2a8-3be0705bb20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:51.828171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9d3f-b48b-466f-a2a8-3be0705bb20a service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:51.828551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9d3f-b48b-466f-a2a8-3be0705bb20a service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:51.828922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9d3f-b48b-466f-a2a8-3be0705bb20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:51.840319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d9d3f-b48b-466f-a2a8-3be0705bb20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:51.842439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d9d3f-b48b-466f-a2a8-3be0705bb20a service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:51.882125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 947/1915] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:52.631541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf79c3-b940-4004-9c86-282db52e8cbe None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:52.634359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3bcf79c3-b940-4004-9c86-282db52e8cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:52.648965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf79c3-b940-4004-9c86-282db52e8cbe None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:52.654190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:52.655111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:52.655392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:52.655668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] PATH_INFO: `/v3/projects/3bcc2144ca4e4ebd981e8e2c0ab073e2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:52.656984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bcc2144ca4e4ebd981e8e2c0ab073e2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:52.659019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:52.734376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bcc2144ca4e4ebd981e8e2c0ab073e2', 'request_id': 'req-57e80613-9593-4f9e-9b3a-c8bd241f928e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:52.740883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bcc2144ca4e4ebd981e8e2c0ab073e2', 'request_id': 'req-57e80613-9593-4f9e-9b3a-c8bd241f928e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:52.747528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bcc2144ca4e4ebd981e8e2c0ab073e2', 'request_id': 'req-57e80613-9593-4f9e-9b3a-c8bd241f928e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:52.763125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] The token cache is being invalidate because project 3bcc2144ca4e4ebd981e8e2c0ab073e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:52.763125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:52.763125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-57e80613-9593-4f9e-9b3a-c8bd241f928e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:52.767367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 969/1916] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:52 2022] DELETE /identity/v3/projects/3bcc2144ca4e4ebd981e8e2c0ab073e2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:52.769095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1050c-3dd3-4bf5-8953-5d02d25b7546 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:52.769456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1050c-3dd3-4bf5-8953-5d02d25b7546 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:52.769829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1050c-3dd3-4bf5-8953-5d02d25b7546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:52.799922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:52.807629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c1f1050c-3dd3-4bf5-8953-5d02d25b7546 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:52.808991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c1f1050c-3dd3-4bf5-8953-5d02d25b7546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:52.895013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 948/1917] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:52.902968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b32175-2537-41e8-be2c-cda0df72f286 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:52.903256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b32175-2537-41e8-be2c-cda0df72f286 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:52.903532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b32175-2537-41e8-be2c-cda0df72f286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:52.926729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-96b32175-2537-41e8-be2c-cda0df72f286 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:52.928390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-96b32175-2537-41e8-be2c-cda0df72f286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:52.937796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 970/1918] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:52.943444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1050c-3dd3-4bf5-8953-5d02d25b7546 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:52.957763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1050c-3dd3-4bf5-8953-5d02d25b7546 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:52.971705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:52.971705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:52.973978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:52.974455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:52.976426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:52.976868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:52.980412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.038331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 949/1919] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:53.040945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b32175-2537-41e8-be2c-cda0df72f286 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:53.059293 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b32175-2537-41e8-be2c-cda0df72f286 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:53.066564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96f5e5-6a77-42fc-ac0c-a4bd945a3dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:53.069496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96f5e5-6a77-42fc-ac0c-a4bd945a3dad admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.070141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96f5e5-6a77-42fc-ac0c-a4bd945a3dad admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.070809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96f5e5-6a77-42fc-ac0c-a4bd945a3dad admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.071865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96f5e5-6a77-42fc-ac0c-a4bd945a3dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:53.074848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96f5e5-6a77-42fc-ac0c-a4bd945a3dad admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.125230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 971/1920] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:53 2022] POST /identity/v3/projects => generated 359 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:53.129100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:53.141291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00257d60-eede-48b3-bf4c-a6f3c854a7c1 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:53.213487 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49df92-84fc-4fa6-bdc5-9d9b78800b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:53.214260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49df92-84fc-4fa6-bdc5-9d9b78800b5b admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.214550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49df92-84fc-4fa6-bdc5-9d9b78800b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.214833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49df92-84fc-4fa6-bdc5-9d9b78800b5b admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.215534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49df92-84fc-4fa6-bdc5-9d9b78800b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:53.217620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49df92-84fc-4fa6-bdc5-9d9b78800b5b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.240584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 950/1921] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:53 2022] POST /identity/v3/users => generated 280 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:53.244566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96f5e5-6a77-42fc-ac0c-a4bd945a3dad admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:53.255879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96f5e5-6a77-42fc-ac0c-a4bd945a3dad admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:53.261044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeea134-280c-4bb0-bf8d-4bfc890292a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:53.261699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeea134-280c-4bb0-bf8d-4bfc890292a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.261984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeea134-280c-4bb0-bf8d-4bfc890292a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.262262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeea134-280c-4bb0-bf8d-4bfc890292a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.262910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeea134-280c-4bb0-bf8d-4bfc890292a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:53.264518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeea134-280c-4bb0-bf8d-4bfc890292a7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.279775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 972/1922] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:49:53.281471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49df92-84fc-4fa6-bdc5-9d9b78800b5b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:53.296212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49df92-84fc-4fa6-bdc5-9d9b78800b5b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:53.302427 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e26f-efb7-4c39-861d-52d0bb3b51f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:53.303698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e26f-efb7-4c39-861d-52d0bb3b51f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.304127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e26f-efb7-4c39-861d-52d0bb3b51f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.304539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e26f-efb7-4c39-861d-52d0bb3b51f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.305291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e26f-efb7-4c39-861d-52d0bb3b51f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:53.307175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e26f-efb7-4c39-861d-52d0bb3b51f5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.320379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 951/1923] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:53.324908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeea134-280c-4bb0-bf8d-4bfc890292a7 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:53.340097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeea134-280c-4bb0-bf8d-4bfc890292a7 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:53.345228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:53.346066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.346416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.346761 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] PATH_INFO: `/v3/projects/696a631872b749cea07c771457b9e0c8/users/e39529034ecf4adc9e00e45193489ae1/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.363981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] RBAC: Authorizing `identity:create_grant(project_id=696a631872b749cea07c771457b9e0c8, user_id=e39529034ecf4adc9e00e45193489ae1, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:53.367024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.383821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 973/1924] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:53 2022] PUT /identity/v3/projects/696a631872b749cea07c771457b9e0c8/users/e39529034ecf4adc9e00e45193489ae1/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:53.394680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a52d79-5489-4b55-a27b-2a9962189f62 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.395173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a52d79-5489-4b55-a27b-2a9962189f62 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.395589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a52d79-5489-4b55-a27b-2a9962189f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.453069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-83a52d79-5489-4b55-a27b-2a9962189f62 None None] MFA Rules not processed for user `e39529034ecf4adc9e00e45193489ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:53.454319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-83a52d79-5489-4b55-a27b-2a9962189f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:53.538258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 952/1925] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:53.578330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:53.580085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:53.631651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:53.650009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1899d92-efdb-446d-9bcb-46ad35c5804e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:53.654502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:53.655940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.656387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.656808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.669339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:53.671874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.685510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 974/1926] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:53.705438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a52d79-5489-4b55-a27b-2a9962189f62 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:53.711512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:53.834578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a52d79-5489-4b55-a27b-2a9962189f62 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:53.847098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:53.849126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.849860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.850492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] PATH_INFO: `/v3/users/e39529034ecf4adc9e00e45193489ae1` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.852990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39529034ecf4adc9e00e45193489ae1)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:53.856663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.934593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:53.936210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:53.951409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39529034ecf4adc9e00e45193489ae1'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:53.957485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114863e-6f80-45ce-9476-323d139379a2 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:53.962064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39529034ecf4adc9e00e45193489ae1'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:53.963548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39529034ecf4adc9e00e45193489ae1'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:53.969344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:53.971783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:53.972063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:53.972338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] PATH_INFO: `/v3/users/b76231620dbf4ff686b5376318c12949` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:53.975338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b76231620dbf4ff686b5376318c12949)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:53.978186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:53.986601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39529034ecf4adc9e00e45193489ae1'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:53.996596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a8588a35-6a1c-449e-afd7-978c1a442cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39529034ecf4adc9e00e45193489ae1'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:54.005901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 953/1927] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:53 2022] DELETE /identity/v3/users/e39529034ecf4adc9e00e45193489ae1 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:54.024463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78aea2-6605-4c18-88b0-0287488965d8 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:54.024463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78aea2-6605-4c18-88b0-0287488965d8 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:54.024463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78aea2-6605-4c18-88b0-0287488965d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:54.040779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7a78aea2-6605-4c18-88b0-0287488965d8 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:54.042706 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7a78aea2-6605-4c18-88b0-0287488965d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:54.082518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b76231620dbf4ff686b5376318c12949'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:54.096396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b76231620dbf4ff686b5376318c12949'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:54.097553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b76231620dbf4ff686b5376318c12949'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:54.117962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b76231620dbf4ff686b5376318c12949'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:54.126888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-46186e78-a021-4457-b281-f812dc1f585f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b76231620dbf4ff686b5376318c12949'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:54.135129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 975/1928] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:53 2022] DELETE /identity/v3/users/b76231620dbf4ff686b5376318c12949 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:54.147295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 954/1929] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:54.149286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888416-84d8-4061-a0b9-8376c89b5e1a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:54.153778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888416-84d8-4061-a0b9-8376c89b5e1a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:54.154275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888416-84d8-4061-a0b9-8376c89b5e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:54.158144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78aea2-6605-4c18-88b0-0287488965d8 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:54.160352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7a78aea2-6605-4c18-88b0-0287488965d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:54.176914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5e888416-84d8-4061-a0b9-8376c89b5e1a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:54.178290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5e888416-84d8-4061-a0b9-8376c89b5e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:54.199883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:54.243358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78aea2-6605-4c18-88b0-0287488965d8 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:54.257166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 976/1930] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:54.267165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e888416-84d8-4061-a0b9-8376c89b5e1a None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:54.307712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:54.308343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:54.308629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:54.308899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:54.323992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:54.326152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:54.339971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 955/1931] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:54.363097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e888416-84d8-4061-a0b9-8376c89b5e1a None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:54.368060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:54.368815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:54.369199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:54.369631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:54.387279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:54.388888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:54.395441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:54.402158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 977/1932] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:55.052540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:55.054046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:55.085570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7842725-3f77-434e-a59a-c3e69d7399e3 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:55.092672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:55.093081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.093363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.093656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] PATH_INFO: `/v3/projects/b292eea3b29848f39946afdcd7c3b905` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.094885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b292eea3b29848f39946afdcd7c3b905)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:55.096965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:55.102039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:55.103755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:55.141212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ed30f-1189-48ef-882a-2c6def07ad60 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:55.149468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:55.160663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.161458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.161773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] PATH_INFO: `/v3/projects/696a631872b749cea07c771457b9e0c8` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.165050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=696a631872b749cea07c771457b9e0c8)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:55.166358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:55.187765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b292eea3b29848f39946afdcd7c3b905', 'request_id': 'req-9c3c4015-7029-4254-9a1b-e3442ac15ecc'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.199607 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b292eea3b29848f39946afdcd7c3b905', 'request_id': 'req-9c3c4015-7029-4254-9a1b-e3442ac15ecc'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.214453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b292eea3b29848f39946afdcd7c3b905', 'request_id': 'req-9c3c4015-7029-4254-9a1b-e3442ac15ecc'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.248429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] The token cache is being invalidate because project b292eea3b29848f39946afdcd7c3b905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:55.248730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.250061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.256300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 956/1933] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:55 2022] DELETE /identity/v3/projects/b292eea3b29848f39946afdcd7c3b905 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:55.257321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:55.261148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:55.278972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696a631872b749cea07c771457b9e0c8', 'request_id': 'req-1e2d2391-d39e-48f3-9e5a-dda14360cccf'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.288934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696a631872b749cea07c771457b9e0c8', 'request_id': 'req-1e2d2391-d39e-48f3-9e5a-dda14360cccf'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.296128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696a631872b749cea07c771457b9e0c8', 'request_id': 'req-1e2d2391-d39e-48f3-9e5a-dda14360cccf'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.311388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] The token cache is being invalidate because project 696a631872b749cea07c771457b9e0c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:55.312026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.312753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e2d2391-d39e-48f3-9e5a-dda14360cccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.314982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 978/1934] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:55 2022] DELETE /identity/v3/projects/696a631872b749cea07c771457b9e0c8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:55.319780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc518-9551-45aa-ad92-4c74b942e9d6 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.320446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc518-9551-45aa-ad92-4c74b942e9d6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.320801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc518-9551-45aa-ad92-4c74b942e9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.339195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c4015-7029-4254-9a1b-e3442ac15ecc admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:55.357036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-83acc518-9551-45aa-ad92-4c74b942e9d6 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:55.358700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-83acc518-9551-45aa-ad92-4c74b942e9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:55.363513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:55.431940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:55.432588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.432869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.433143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] PATH_INFO: `/v3/users/21ad3885e4bb4ec3ae5b6c13598e813c` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.434386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ad3885e4bb4ec3ae5b6c13598e813c)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:55.436403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:55.474773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 979/1935] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:55.484837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad282b-6aeb-4e7f-b558-3cc1c489eb69 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.485580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad282b-6aeb-4e7f-b558-3cc1c489eb69 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.486033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad282b-6aeb-4e7f-b558-3cc1c489eb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.493756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ad3885e4bb4ec3ae5b6c13598e813c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.506707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ad3885e4bb4ec3ae5b6c13598e813c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.507337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ad3885e4bb4ec3ae5b6c13598e813c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.534687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4aad282b-6aeb-4e7f-b558-3cc1c489eb69 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:55.535967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4aad282b-6aeb-4e7f-b558-3cc1c489eb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:55.540988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ad3885e4bb4ec3ae5b6c13598e813c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.547960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 980/1936] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:55.550391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ad3885e4bb4ec3ae5b6c13598e813c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:55.553327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad282b-6aeb-4e7f-b558-3cc1c489eb69 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:55.561205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 957/1937] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:55 2022] DELETE /identity/v3/users/21ad3885e4bb4ec3ae5b6c13598e813c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:55.567645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad282b-6aeb-4e7f-b558-3cc1c489eb69 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:55.649199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:55.650015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.650390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.650756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.651820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:55.652223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:55.655102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:55.674949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 981/1938] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:49:55.680074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:55.696168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e438fb-11d7-410f-a166-ff7dd0eb5538 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:55.702035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e74b9a-2ef9-4b51-a388-a8d2ad37f8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:55.703253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e74b9a-2ef9-4b51-a388-a8d2ad37f8bc admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.703871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e74b9a-2ef9-4b51-a388-a8d2ad37f8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.704332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e74b9a-2ef9-4b51-a388-a8d2ad37f8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.705249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e74b9a-2ef9-4b51-a388-a8d2ad37f8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:55.707874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e74b9a-2ef9-4b51-a388-a8d2ad37f8bc admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:55.765554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 958/1939] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:55 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:55.770002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:55.782319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67508de-982a-4f47-8507-9de89a86d925 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:55.793415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:55.873783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a905d32-6dd1-4527-8955-7eb312b985f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:55.874584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905d32-6dd1-4527-8955-7eb312b985f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.874967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905d32-6dd1-4527-8955-7eb312b985f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.875385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905d32-6dd1-4527-8955-7eb312b985f8 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.876299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a905d32-6dd1-4527-8955-7eb312b985f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:55.878385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a905d32-6dd1-4527-8955-7eb312b985f8 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:55.929687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 982/1940] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:55 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:55.933999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e74b9a-2ef9-4b51-a388-a8d2ad37f8bc admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:55.958631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e74b9a-2ef9-4b51-a388-a8d2ad37f8bc admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:55.968380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04bf11-ab72-4e5b-995e-a36d274d0ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:55.969136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04bf11-ab72-4e5b-995e-a36d274d0ecc admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:55.969442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04bf11-ab72-4e5b-995e-a36d274d0ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:55.969756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da04bf11-ab72-4e5b-995e-a36d274d0ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:55.970421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04bf11-ab72-4e5b-995e-a36d274d0ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:55.972167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da04bf11-ab72-4e5b-995e-a36d274d0ecc admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:55.997492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 959/1941] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:56.001171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a905d32-6dd1-4527-8955-7eb312b985f8 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.012835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a905d32-6dd1-4527-8955-7eb312b985f8 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.019057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed855c-b381-4c8e-a2cf-543f5d7a7c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.019915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed855c-b381-4c8e-a2cf-543f5d7a7c5e admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.020423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed855c-b381-4c8e-a2cf-543f5d7a7c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.020838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed855c-b381-4c8e-a2cf-543f5d7a7c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.021749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed855c-b381-4c8e-a2cf-543f5d7a7c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.024071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed855c-b381-4c8e-a2cf-543f5d7a7c5e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.041136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 983/1942] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:56.046143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04bf11-ab72-4e5b-995e-a36d274d0ecc admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.064218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da04bf11-ab72-4e5b-995e-a36d274d0ecc admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.072433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a848c-978d-4c15-9a92-a2fe62084bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.073360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a848c-978d-4c15-9a92-a2fe62084bb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.073991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a848c-978d-4c15-9a92-a2fe62084bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.074373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a848c-978d-4c15-9a92-a2fe62084bb4 admin admin] PATH_INFO: `/v3/projects/4a31ab11fa4c442d93a2075a29d05a4d/users/47ec3a8bf2fa487a9304745003e428d9/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.098683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a848c-978d-4c15-9a92-a2fe62084bb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a31ab11fa4c442d93a2075a29d05a4d, user_id=47ec3a8bf2fa487a9304745003e428d9, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.101858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a848c-978d-4c15-9a92-a2fe62084bb4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.125884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 960/1943] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:56 2022] PUT /identity/v3/projects/4a31ab11fa4c442d93a2075a29d05a4d/users/47ec3a8bf2fa487a9304745003e428d9/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:56.134889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd6d8d-f0a4-4f47-965b-fa4d945d3138 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.135992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd6d8d-f0a4-4f47-965b-fa4d945d3138 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.136327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd6d8d-f0a4-4f47-965b-fa4d945d3138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.206572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-eafd6d8d-f0a4-4f47-965b-fa4d945d3138 None None] MFA Rules not processed for user `47ec3a8bf2fa487a9304745003e428d9`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:56.208473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-eafd6d8d-f0a4-4f47-965b-fa4d945d3138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:56.377543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 984/1944] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:49:56.378796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a848c-978d-4c15-9a92-a2fe62084bb4 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.432426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:56.466030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a848c-978d-4c15-9a92-a2fe62084bb4 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.471403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db8ed9-19af-48a0-bcd1-ebf4d5cc17a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.472248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8ed9-19af-48a0-bcd1-ebf4d5cc17a1 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.472709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8ed9-19af-48a0-bcd1-ebf4d5cc17a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.472977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db8ed9-19af-48a0-bcd1-ebf4d5cc17a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.474376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db8ed9-19af-48a0-bcd1-ebf4d5cc17a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.476384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db8ed9-19af-48a0-bcd1-ebf4d5cc17a1 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.484644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd6d8d-f0a4-4f47-965b-fa4d945d3138 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.486406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-eafd6d8d-f0a4-4f47-965b-fa4d945d3138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:56.504913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd6d8d-f0a4-4f47-965b-fa4d945d3138 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.511814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.514953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.516021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.516320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] PATH_INFO: `/v3/projects/89cbd3e76f364dd995fe9bbe6720600c` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.517918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89cbd3e76f364dd995fe9bbe6720600c)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.521145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.529242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 961/1945] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:56.536871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db8ed9-19af-48a0-bcd1-ebf4d5cc17a1 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.606294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89cbd3e76f364dd995fe9bbe6720600c', 'request_id': 'req-e8b99a0c-823a-4c62-bda6-5591aac59255'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:56.613496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89cbd3e76f364dd995fe9bbe6720600c', 'request_id': 'req-e8b99a0c-823a-4c62-bda6-5591aac59255'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:56.621928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89cbd3e76f364dd995fe9bbe6720600c', 'request_id': 'req-e8b99a0c-823a-4c62-bda6-5591aac59255'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:56.628833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db8ed9-19af-48a0-bcd1-ebf4d5cc17a1 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.640296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] The token cache is being invalidate because project 89cbd3e76f364dd995fe9bbe6720600c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:49:56.640592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:56.641554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e8b99a0c-823a-4c62-bda6-5591aac59255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:56.643658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 985/1946] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:56 2022] DELETE /identity/v3/projects/89cbd3e76f364dd995fe9bbe6720600c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:56.650060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed368507-292f-4412-bb7b-c42ba27c05aa None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.651141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed368507-292f-4412-bb7b-c42ba27c05aa None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.651527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed368507-292f-4412-bb7b-c42ba27c05aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.661450 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:56.688186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ed368507-292f-4412-bb7b-c42ba27c05aa None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:56.689971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ed368507-292f-4412-bb7b-c42ba27c05aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:56.707070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.708404 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.708785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.709156 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.712263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.714086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.744335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 962/1947] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:56.749323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.751047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:56.764267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba159a-45e8-4563-af8e-29adc62c7eb2 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.769510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187926f8-0986-44c1-899a-9f4ce6b4adf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.770141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187926f8-0986-44c1-899a-9f4ce6b4adf2 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.770486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187926f8-0986-44c1-899a-9f4ce6b4adf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.770761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187926f8-0986-44c1-899a-9f4ce6b4adf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.772483 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187926f8-0986-44c1-899a-9f4ce6b4adf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.773317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187926f8-0986-44c1-899a-9f4ce6b4adf2 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.782855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 986/1948] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:56.789456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 963/1949] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:56.791568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e6ee7-df5d-4a68-be10-f8b8503999d7 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.791919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e6ee7-df5d-4a68-be10-f8b8503999d7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.792196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e6ee7-df5d-4a68-be10-f8b8503999d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.793994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187926f8-0986-44c1-899a-9f4ce6b4adf2 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.815509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-eb8e6ee7-df5d-4a68-be10-f8b8503999d7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:56.816745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-eb8e6ee7-df5d-4a68-be10-f8b8503999d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:56.819712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187926f8-0986-44c1-899a-9f4ce6b4adf2 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.827041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff3af0-d215-44d0-bf0a-33a2ba1be970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.827916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3af0-d215-44d0-bf0a-33a2ba1be970 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.828371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3af0-d215-44d0-bf0a-33a2ba1be970 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.828920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3af0-d215-44d0-bf0a-33a2ba1be970 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.830760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff3af0-d215-44d0-bf0a-33a2ba1be970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.834860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 987/1950] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:56.835999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e6ee7-df5d-4a68-be10-f8b8503999d7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.836532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff3af0-d215-44d0-bf0a-33a2ba1be970 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.846132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e6ee7-df5d-4a68-be10-f8b8503999d7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.857637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 964/1951] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:56.858409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.859061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.859344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.859613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.860341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:49:56.860823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.862481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.862748 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff3af0-d215-44d0-bf0a-33a2ba1be970 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.877920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff3af0-d215-44d0-bf0a-33a2ba1be970 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.881289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 988/1952] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:49:56.886968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.887556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc6e5c-161e-4fe4-85ca-b32ee5a3ebd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.888540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc6e5c-161e-4fe4-85ca-b32ee5a3ebd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.888978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc6e5c-161e-4fe4-85ca-b32ee5a3ebd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.889406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc6e5c-161e-4fe4-85ca-b32ee5a3ebd7 admin admin] PATH_INFO: `/v3/projects/881e89929e274428992c863800b066e2/users/30ba672ac28d423d9ce4c92c4bb7a7d8/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.906387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afd387-3d1d-472a-883a-e2e50a05c776 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:56.911286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc6e5c-161e-4fe4-85ca-b32ee5a3ebd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=881e89929e274428992c863800b066e2, user_id=30ba672ac28d423d9ce4c92c4bb7a7d8, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.913154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4503b-3bc4-433e-a7fe-4f9ae0153751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:56.914048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4503b-3bc4-433e-a7fe-4f9ae0153751 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.914177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc6e5c-161e-4fe4-85ca-b32ee5a3ebd7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.914598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4503b-3bc4-433e-a7fe-4f9ae0153751 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.915022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4503b-3bc4-433e-a7fe-4f9ae0153751 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.915818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4503b-3bc4-433e-a7fe-4f9ae0153751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:56.918109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4503b-3bc4-433e-a7fe-4f9ae0153751 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:56.938313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 965/1953] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:56 2022] PUT /identity/v3/projects/881e89929e274428992c863800b066e2/users/30ba672ac28d423d9ce4c92c4bb7a7d8/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:56.949099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2f46-e41d-4653-80cb-3d3cc9b71508 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:56.949398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2f46-e41d-4653-80cb-3d3cc9b71508 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:56.949725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2f46-e41d-4653-80cb-3d3cc9b71508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:56.960620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 989/1954] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/projects => generated 381 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:56.964648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4503b-3bc4-433e-a7fe-4f9ae0153751 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:56.975033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:56.992868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8b0b2f46-e41d-4653-80cb-3d3cc9b71508 None None] MFA Rules not processed for user `30ba672ac28d423d9ce4c92c4bb7a7d8`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:56.994376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8b0b2f46-e41d-4653-80cb-3d3cc9b71508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:57.057156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4503b-3bc4-433e-a7fe-4f9ae0153751 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.062184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb207f-1f9e-4d68-9d61-40db7caff5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.062809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb207f-1f9e-4d68-9d61-40db7caff5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.063082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb207f-1f9e-4d68-9d61-40db7caff5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.063351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb207f-1f9e-4d68-9d61-40db7caff5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.063987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb207f-1f9e-4d68-9d61-40db7caff5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.065977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cb207f-1f9e-4d68-9d61-40db7caff5e0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.091300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 990/1955] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:57.096092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb207f-1f9e-4d68-9d61-40db7caff5e0 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.102948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 966/1956] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:57.107148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b2f46-e41d-4653-80cb-3d3cc9b71508 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.117526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cb207f-1f9e-4d68-9d61-40db7caff5e0 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.120864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b2f46-e41d-4653-80cb-3d3cc9b71508 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.122980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4320-7a78-410e-9be4-4dcdc01ad3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.123866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4320-7a78-410e-9be4-4dcdc01ad3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.124154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4320-7a78-410e-9be4-4dcdc01ad3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.124432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc4320-7a78-410e-9be4-4dcdc01ad3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.125084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc4320-7a78-410e-9be4-4dcdc01ad3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.126331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c98e6e-3df2-48e9-8908-0a00765b11c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.127166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c98e6e-3df2-48e9-8908-0a00765b11c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.127289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc4320-7a78-410e-9be4-4dcdc01ad3c2 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.127578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c98e6e-3df2-48e9-8908-0a00765b11c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.127965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c98e6e-3df2-48e9-8908-0a00765b11c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.128870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c98e6e-3df2-48e9-8908-0a00765b11c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.133007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c98e6e-3df2-48e9-8908-0a00765b11c0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.139185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 991/1957] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:57.143302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4320-7a78-410e-9be4-4dcdc01ad3c2 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.158821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc4320-7a78-410e-9be4-4dcdc01ad3c2 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.164804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046f67-a9eb-4518-aa8d-569632f0d135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.165439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046f67-a9eb-4518-aa8d-569632f0d135 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.165781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046f67-a9eb-4518-aa8d-569632f0d135 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.166064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046f67-a9eb-4518-aa8d-569632f0d135 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.166713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76046f67-a9eb-4518-aa8d-569632f0d135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.168517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76046f67-a9eb-4518-aa8d-569632f0d135 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.172186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 967/1958] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:57 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:57.173469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c98e6e-3df2-48e9-8908-0a00765b11c0 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.184274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 992/1959] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:49:57.188654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046f67-a9eb-4518-aa8d-569632f0d135 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.200795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046f67-a9eb-4518-aa8d-569632f0d135 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.236041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:57.267716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c98e6e-3df2-48e9-8908-0a00765b11c0 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.275285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39969b0-a861-44d1-8067-1addba214a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.275897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39969b0-a861-44d1-8067-1addba214a74 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.276180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39969b0-a861-44d1-8067-1addba214a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.276456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39969b0-a861-44d1-8067-1addba214a74 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.277092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39969b0-a861-44d1-8067-1addba214a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.283095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39969b0-a861-44d1-8067-1addba214a74 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.285524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a6c1-e790-41f9-8064-095bfdaa9e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.287023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a6c1-e790-41f9-8064-095bfdaa9e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.295640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a6c1-e790-41f9-8064-095bfdaa9e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.296140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a6c1-e790-41f9-8064-095bfdaa9e84 admin admin] PATH_INFO: `/v3/projects/ed700f7691f142db833340289c873f1d/users/2a51957120244295b7d3304da2b62d4f/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.317919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 968/1960] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:57 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:49:57.321509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39969b0-a861-44d1-8067-1addba214a74 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.335220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39969b0-a861-44d1-8067-1addba214a74 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.351529 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093a6c1-e790-41f9-8064-095bfdaa9e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed700f7691f142db833340289c873f1d, user_id=2a51957120244295b7d3304da2b62d4f, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.354277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093a6c1-e790-41f9-8064-095bfdaa9e84 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.357363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecc0ee-c5bf-432e-b3e9-f58e6ea2c95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.358020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc0ee-c5bf-432e-b3e9-f58e6ea2c95e admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.358303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc0ee-c5bf-432e-b3e9-f58e6ea2c95e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.358577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecc0ee-c5bf-432e-b3e9-f58e6ea2c95e admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.360476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecc0ee-c5bf-432e-b3e9-f58e6ea2c95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.360846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecc0ee-c5bf-432e-b3e9-f58e6ea2c95e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.382801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 993/1961] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:57 2022] PUT /identity/v3/projects/ed700f7691f142db833340289c873f1d/users/2a51957120244295b7d3304da2b62d4f/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:57.394232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3675394-4bd6-4c67-a8a6-f1bf06c3ea52 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.394821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3675394-4bd6-4c67-a8a6-f1bf06c3ea52 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.395648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3675394-4bd6-4c67-a8a6-f1bf06c3ea52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.398164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 969/1962] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:57.401301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecc0ee-c5bf-432e-b3e9-f58e6ea2c95e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.420197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecc0ee-c5bf-432e-b3e9-f58e6ea2c95e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.510806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f3675394-4bd6-4c67-a8a6-f1bf06c3ea52 None None] MFA Rules not processed for user `2a51957120244295b7d3304da2b62d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:57.512237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f3675394-4bd6-4c67-a8a6-f1bf06c3ea52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:57.526324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:57.537462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7938e-f4fd-4150-b466-6acb139113c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.538224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7938e-f4fd-4150-b466-6acb139113c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.539043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7938e-f4fd-4150-b466-6acb139113c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.539599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7938e-f4fd-4150-b466-6acb139113c3 admin admin] PATH_INFO: `/v3/projects/2c6e632dc08d40469fbdee90a93efcfd/users/aa7fd212c31341ce926ee60ff1451e0a/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.564440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7938e-f4fd-4150-b466-6acb139113c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6e632dc08d40469fbdee90a93efcfd, user_id=aa7fd212c31341ce926ee60ff1451e0a, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.568217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7938e-f4fd-4150-b466-6acb139113c3 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.600183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 970/1963] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:57 2022] PUT /identity/v3/projects/2c6e632dc08d40469fbdee90a93efcfd/users/aa7fd212c31341ce926ee60ff1451e0a/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:57.603412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7938e-f4fd-4150-b466-6acb139113c3 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.625621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 994/1964] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:57.631268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3675394-4bd6-4c67-a8a6-f1bf06c3ea52 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.649051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3675394-4bd6-4c67-a8a6-f1bf06c3ea52 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.708702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7938e-f4fd-4150-b466-6acb139113c3 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.714048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c68b9-5fa3-48cc-9665-dfb1c8ee6105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.714771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c68b9-5fa3-48cc-9665-dfb1c8ee6105 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.715047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c68b9-5fa3-48cc-9665-dfb1c8ee6105 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.715442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c68b9-5fa3-48cc-9665-dfb1c8ee6105 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.715960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c68b9-5fa3-48cc-9665-dfb1c8ee6105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.717759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c68b9-5fa3-48cc-9665-dfb1c8ee6105 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.731465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 971/1965] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:57.735601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c68b9-5fa3-48cc-9665-dfb1c8ee6105 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.737994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786074e-1be8-4531-9e68-73bf1de00df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.738811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786074e-1be8-4531-9e68-73bf1de00df9 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.739782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786074e-1be8-4531-9e68-73bf1de00df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.740205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786074e-1be8-4531-9e68-73bf1de00df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.740989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786074e-1be8-4531-9e68-73bf1de00df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.744354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786074e-1be8-4531-9e68-73bf1de00df9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.748773 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c68b9-5fa3-48cc-9665-dfb1c8ee6105 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.756945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b328f12-e4d5-4148-92d1-74bc47bac57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.757654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b328f12-e4d5-4148-92d1-74bc47bac57d admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.757940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b328f12-e4d5-4148-92d1-74bc47bac57d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.758888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b328f12-e4d5-4148-92d1-74bc47bac57d admin admin] PATH_INFO: `/v3/projects/2c6e632dc08d40469fbdee90a93efcfd/users/aa7fd212c31341ce926ee60ff1451e0a/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.761076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b328f12-e4d5-4148-92d1-74bc47bac57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6e632dc08d40469fbdee90a93efcfd, user_id=aa7fd212c31341ce926ee60ff1451e0a, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.765121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b328f12-e4d5-4148-92d1-74bc47bac57d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.776454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 995/1966] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:49:57 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:57.780457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786074e-1be8-4531-9e68-73bf1de00df9 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.782110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 972/1967] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:57 2022] PUT /identity/v3/projects/2c6e632dc08d40469fbdee90a93efcfd/users/aa7fd212c31341ce926ee60ff1451e0a/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:57.794769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a238bc9-1199-4798-80fd-46197cad2259 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.795295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a238bc9-1199-4798-80fd-46197cad2259 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.795917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a238bc9-1199-4798-80fd-46197cad2259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.825033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:57.836646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-5a238bc9-1199-4798-80fd-46197cad2259 None None] MFA Rules not processed for user `aa7fd212c31341ce926ee60ff1451e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:57.838619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5a238bc9-1199-4798-80fd-46197cad2259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:57.864466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786074e-1be8-4531-9e68-73bf1de00df9 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.869669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83008431-56d5-4bc4-b387-a71e94d0c337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.870296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83008431-56d5-4bc4-b387-a71e94d0c337 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.870696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83008431-56d5-4bc4-b387-a71e94d0c337 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.870976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83008431-56d5-4bc4-b387-a71e94d0c337 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.871619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83008431-56d5-4bc4-b387-a71e94d0c337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.873570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83008431-56d5-4bc4-b387-a71e94d0c337 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.904628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 996/1968] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:49:57 2022] POST /identity/v3/users => generated 297 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:49:57.905476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83008431-56d5-4bc4-b387-a71e94d0c337 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.927695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83008431-56d5-4bc4-b387-a71e94d0c337 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.933271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f109f9-ba46-4921-b921-f1834ad06cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.933928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f109f9-ba46-4921-b921-f1834ad06cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.934210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f109f9-ba46-4921-b921-f1834ad06cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.934486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f109f9-ba46-4921-b921-f1834ad06cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.935019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 973/1969] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:57.935128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f109f9-ba46-4921-b921-f1834ad06cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.936790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f109f9-ba46-4921-b921-f1834ad06cd3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:57.946909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a238bc9-1199-4798-80fd-46197cad2259 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.948377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5a238bc9-1199-4798-80fd-46197cad2259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:57.953654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 997/1970] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:49:57.955526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f109f9-ba46-4921-b921-f1834ad06cd3 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:57.966499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f109f9-ba46-4921-b921-f1834ad06cd3 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:57.971679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b342b-a5c5-4d0f-8e17-9ab51cced55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:57.972358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b342b-a5c5-4d0f-8e17-9ab51cced55c admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:57.972637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b342b-a5c5-4d0f-8e17-9ab51cced55c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:57.972913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b342b-a5c5-4d0f-8e17-9ab51cced55c admin admin] PATH_INFO: `/v3/projects/e28feb29f76244d0837df51576d5e18b/users/079184eddf8e44768fbf64b468bb2a24/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:57.989670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b342b-a5c5-4d0f-8e17-9ab51cced55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28feb29f76244d0837df51576d5e18b, user_id=079184eddf8e44768fbf64b468bb2a24, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:57.992419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b342b-a5c5-4d0f-8e17-9ab51cced55c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:58.016994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 998/1971] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:57 2022] PUT /identity/v3/projects/e28feb29f76244d0837df51576d5e18b/users/079184eddf8e44768fbf64b468bb2a24/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:49:58.017766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b342b-a5c5-4d0f-8e17-9ab51cced55c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:58.070403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:58.079590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a238bc9-1199-4798-80fd-46197cad2259 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:58.110242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b342b-a5c5-4d0f-8e17-9ab51cced55c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:58.116223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72368c7-b8aa-41a9-a3b8-8977b9d0c369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:58.116989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72368c7-b8aa-41a9-a3b8-8977b9d0c369 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:58.117271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72368c7-b8aa-41a9-a3b8-8977b9d0c369 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:58.117753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72368c7-b8aa-41a9-a3b8-8977b9d0c369 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:58.118640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72368c7-b8aa-41a9-a3b8-8977b9d0c369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:58.121690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72368c7-b8aa-41a9-a3b8-8977b9d0c369 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:58.137481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 999/1972] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:49:58.141248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72368c7-b8aa-41a9-a3b8-8977b9d0c369 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:58.157678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72368c7-b8aa-41a9-a3b8-8977b9d0c369 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:58.167904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cbe9c-26b5-4d82-bfe7-91ffd8d7d462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:58.179577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cbe9c-26b5-4d82-bfe7-91ffd8d7d462 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:58.179884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cbe9c-26b5-4d82-bfe7-91ffd8d7d462 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:58.180447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cbe9c-26b5-4d82-bfe7-91ffd8d7d462 admin admin] PATH_INFO: `/v3/projects/e28feb29f76244d0837df51576d5e18b/users/079184eddf8e44768fbf64b468bb2a24/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:58.186958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cbe9c-26b5-4d82-bfe7-91ffd8d7d462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28feb29f76244d0837df51576d5e18b, user_id=079184eddf8e44768fbf64b468bb2a24, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:58.192942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cbe9c-26b5-4d82-bfe7-91ffd8d7d462 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:58.198444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:58.198974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:58.199527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:58.199985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:58.202452 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:58.219172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1000/1973] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:49:58 2022] PUT /identity/v3/projects/e28feb29f76244d0837df51576d5e18b/users/079184eddf8e44768fbf64b468bb2a24/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:58.230947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db6d8-fbd2-4736-9b93-9f997fd5d8df None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:58.231464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db6d8-fbd2-4736-9b93-9f997fd5d8df None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:58.231964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db6d8-fbd2-4736-9b93-9f997fd5d8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:58.274339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9c5db6d8-fbd2-4736-9b93-9f997fd5d8df None None] MFA Rules not processed for user `079184eddf8e44768fbf64b468bb2a24`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:58.275896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9c5db6d8-fbd2-4736-9b93-9f997fd5d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:58.349903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:58.351901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:58.363140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:58.366628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 974/1974] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:58.401243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1001/1975] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:58.414774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:58.416321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:58.430834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892b17a-a5ce-4648-bce1-1219380b8072 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:58.505816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30a49c-3bb7-40b5-81ff-cb00c734ada4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:58.506471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30a49c-3bb7-40b5-81ff-cb00c734ada4 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:58.506802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30a49c-3bb7-40b5-81ff-cb00c734ada4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:58.507127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30a49c-3bb7-40b5-81ff-cb00c734ada4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:58.526955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30a49c-3bb7-40b5-81ff-cb00c734ada4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:58.529228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30a49c-3bb7-40b5-81ff-cb00c734ada4 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:58.561989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 975/1976] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:58.800405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5db6d8-fbd2-4736-9b93-9f997fd5d8df None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:58.811661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5db6d8-fbd2-4736-9b93-9f997fd5d8df None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:58.819406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4c7ea4-4670-42cb-8893-0894d452782b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:58.820316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c7ea4-4670-42cb-8893-0894d452782b service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:58.820838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c7ea4-4670-42cb-8893-0894d452782b service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:58.822533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4c7ea4-4670-42cb-8893-0894d452782b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:58.908248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c7ea4-4670-42cb-8893-0894d452782b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:58.910047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4c7ea4-4670-42cb-8893-0894d452782b service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:58.927936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1002/1977] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:49:59.328739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30a49c-3bb7-40b5-81ff-cb00c734ada4 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:59.393356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30a49c-3bb7-40b5-81ff-cb00c734ada4 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:59.398503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:59.399147 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:59.399454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:59.399810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] PATH_INFO: `/v3/users/2a51957120244295b7d3304da2b62d4f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:59.401195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a51957120244295b7d3304da2b62d4f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:59.403530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:59.405159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:49:59.454151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a51957120244295b7d3304da2b62d4f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:59.461279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a51957120244295b7d3304da2b62d4f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:59.461930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a51957120244295b7d3304da2b62d4f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:59.474228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a51957120244295b7d3304da2b62d4f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:59.481553 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a51957120244295b7d3304da2b62d4f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:49:59.487497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 976/1978] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:49:59 2022] DELETE /identity/v3/users/2a51957120244295b7d3304da2b62d4f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:49:59.492489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3823f70-26a9-44e0-873e-2513e14b40a5 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:59.492994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3823f70-26a9-44e0-873e-2513e14b40a5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:59.493093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3823f70-26a9-44e0-873e-2513e14b40a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:59.511703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a3823f70-26a9-44e0-873e-2513e14b40a5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:49:59.512896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a3823f70-26a9-44e0-873e-2513e14b40a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:59.571720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1003/1979] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:49:59.581426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:59.582852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:59.642110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c090e-995d-4735-817e-8a9512bbff0b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:59.645726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec93e47-1d08-4e81-b99e-37ecd511fb9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:59.646324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93e47-1d08-4e81-b99e-37ecd511fb9d service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:59.646601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93e47-1d08-4e81-b99e-37ecd511fb9d service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:59.646873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93e47-1d08-4e81-b99e-37ecd511fb9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:59.656444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec93e47-1d08-4e81-b99e-37ecd511fb9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:59.657885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec93e47-1d08-4e81-b99e-37ecd511fb9d service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:49:59.667497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 977/1980] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:49:59.930099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3823f70-26a9-44e0-873e-2513e14b40a5 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:49:59.931596 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a3823f70-26a9-44e0-873e-2513e14b40a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:49:59.944992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3823f70-26a9-44e0-873e-2513e14b40a5 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:49:59.949549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:49:59.950244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:49:59.950480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:49:59.950738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] PATH_INFO: `/v3/projects/ed700f7691f142db833340289c873f1d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:49:59.951867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed700f7691f142db833340289c873f1d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:49:59.953808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:00.004953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed700f7691f142db833340289c873f1d', 'request_id': 'req-51f91024-5a5e-4896-9058-bc1fa17a0aa0'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.019091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed700f7691f142db833340289c873f1d', 'request_id': 'req-51f91024-5a5e-4896-9058-bc1fa17a0aa0'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.026309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed700f7691f142db833340289c873f1d', 'request_id': 'req-51f91024-5a5e-4896-9058-bc1fa17a0aa0'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.041488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] The token cache is being invalidate because project ed700f7691f142db833340289c873f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:00.041830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.042459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.043972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1004/1981] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:49:59 2022] DELETE /identity/v3/projects/ed700f7691f142db833340289c873f1d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:00.047979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec93e47-1d08-4e81-b99e-37ecd511fb9d service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:00.058505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec93e47-1d08-4e81-b99e-37ecd511fb9d service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:00.122749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:00.123586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:00.124027 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:00.124447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] PATH_INFO: `/v3/users/079184eddf8e44768fbf64b468bb2a24` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:00.125855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=079184eddf8e44768fbf64b468bb2a24)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:00.127953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:00.173526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:00.206835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079184eddf8e44768fbf64b468bb2a24'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.215488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079184eddf8e44768fbf64b468bb2a24'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.216291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079184eddf8e44768fbf64b468bb2a24'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.230589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079184eddf8e44768fbf64b468bb2a24'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.239118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079184eddf8e44768fbf64b468bb2a24'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.248837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 978/1982] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:00 2022] DELETE /identity/v3/users/079184eddf8e44768fbf64b468bb2a24 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:00.295031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:00.296486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:00.312408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:00.363159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f91024-5a5e-4896-9058-bc1fa17a0aa0 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:00.428422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc87d3-94a0-4032-bc71-17c38eeb8aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:00.429169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc87d3-94a0-4032-bc71-17c38eeb8aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:00.429517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc87d3-94a0-4032-bc71-17c38eeb8aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:00.430258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc87d3-94a0-4032-bc71-17c38eeb8aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:00.432173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-26dc87d3-94a0-4032-bc71-17c38eeb8aa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:00.509574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc87d3-94a0-4032-bc71-17c38eeb8aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:00.511430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc87d3-94a0-4032-bc71-17c38eeb8aa7 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:00.524302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1005/1983] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:00.656959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:00.658900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:00.745644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062578f2-e98b-412e-bf9b-ac9c3b860085 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:00.750238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:00.750867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:00.751146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:00.751418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] PATH_INFO: `/v3/projects/e28feb29f76244d0837df51576d5e18b` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:00.752552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e28feb29f76244d0837df51576d5e18b)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:00.755235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:00.805368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e28feb29f76244d0837df51576d5e18b', 'request_id': 'req-cba0b5b1-4d70-4159-b785-e06b0ee760d0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.811415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e28feb29f76244d0837df51576d5e18b', 'request_id': 'req-cba0b5b1-4d70-4159-b785-e06b0ee760d0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.817913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e28feb29f76244d0837df51576d5e18b', 'request_id': 'req-cba0b5b1-4d70-4159-b785-e06b0ee760d0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.836503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] The token cache is being invalidate because project e28feb29f76244d0837df51576d5e18b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:00.837100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.837980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cba0b5b1-4d70-4159-b785-e06b0ee760d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:00.839964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 979/1984] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:00 2022] DELETE /identity/v3/projects/e28feb29f76244d0837df51576d5e18b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:00.845581 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ff48-770d-470c-8cff-e8de982491cf None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:00.845960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ff48-770d-470c-8cff-e8de982491cf None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:00.846310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903ff48-770d-470c-8cff-e8de982491cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:00.866325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-c903ff48-770d-470c-8cff-e8de982491cf None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:00.867530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c903ff48-770d-470c-8cff-e8de982491cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:00.934028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1006/1985] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:00.943181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc756df3-6c6f-4f00-8a46-25c08fd56595 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:00.943471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc756df3-6c6f-4f00-8a46-25c08fd56595 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:00.943740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc756df3-6c6f-4f00-8a46-25c08fd56595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:00.968307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-dc756df3-6c6f-4f00-8a46-25c08fd56595 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:00.970487 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dc756df3-6c6f-4f00-8a46-25c08fd56595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:00.980133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 980/1986] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:00.984455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903ff48-770d-470c-8cff-e8de982491cf None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:00.996288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903ff48-770d-470c-8cff-e8de982491cf None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.001726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.002400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.002742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.003015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.003656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:01.003946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.005528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.019803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1007/1987] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:01.022013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc756df3-6c6f-4f00-8a46-25c08fd56595 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.032087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc756df3-6c6f-4f00-8a46-25c08fd56595 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.037248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.037910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.038194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.038467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.039096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.041050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.077056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 981/1988] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:01 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:01.078345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.089135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4651d7-3456-48de-9a6a-0e5d5f031669 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.094316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:01.147991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1e92f-e0fc-498a-8cf7-5c26b6449710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.148608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1e92f-e0fc-498a-8cf7-5c26b6449710 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.148890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1e92f-e0fc-498a-8cf7-5c26b6449710 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.149164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1e92f-e0fc-498a-8cf7-5c26b6449710 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.149823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1e92f-e0fc-498a-8cf7-5c26b6449710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.151751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1e92f-e0fc-498a-8cf7-5c26b6449710 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.164274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.166264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:01.171842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:01.178348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1008/1989] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:01 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:01.179673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1e92f-e0fc-498a-8cf7-5c26b6449710 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.196413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1e92f-e0fc-498a-8cf7-5c26b6449710 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.201432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71a4e3-e0ed-4891-bb8e-f7f6b771f5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.202081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71a4e3-e0ed-4891-bb8e-f7f6b771f5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.203785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71a4e3-e0ed-4891-bb8e-f7f6b771f5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.203785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d71a4e3-e0ed-4891-bb8e-f7f6b771f5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.203785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71a4e3-e0ed-4891-bb8e-f7f6b771f5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.204843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d71a4e3-e0ed-4891-bb8e-f7f6b771f5a8 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.220436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1009/1990] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:01.225137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71a4e3-e0ed-4891-bb8e-f7f6b771f5a8 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.235372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d71a4e3-e0ed-4891-bb8e-f7f6b771f5a8 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.241550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385bc62-0067-4fad-b612-a66995a8379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.242122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385bc62-0067-4fad-b612-a66995a8379b admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.242403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385bc62-0067-4fad-b612-a66995a8379b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.242674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385bc62-0067-4fad-b612-a66995a8379b admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.243461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385bc62-0067-4fad-b612-a66995a8379b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.245999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385bc62-0067-4fad-b612-a66995a8379b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.249941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836bb60-9767-4fb2-9a00-0c6c8492f1aa admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.255220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.256081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.256548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.256756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1010/1991] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:01.257132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.260711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:01.261128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385bc62-0067-4fad-b612-a66995a8379b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.272423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385bc62-0067-4fad-b612-a66995a8379b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.277292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc72766-6f14-44d9-b80b-dca4a41dbf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.277994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc72766-6f14-44d9-b80b-dca4a41dbf5a admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.278277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc72766-6f14-44d9-b80b-dca4a41dbf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.278557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc72766-6f14-44d9-b80b-dca4a41dbf5a admin admin] PATH_INFO: `/v3/projects/895a56a4fb9e48cd8635792dcf3d4cf9/users/26fee53a0fc34a70be9eef9d9f117f57/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.295156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc72766-6f14-44d9-b80b-dca4a41dbf5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=895a56a4fb9e48cd8635792dcf3d4cf9, user_id=26fee53a0fc34a70be9eef9d9f117f57, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.298085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc72766-6f14-44d9-b80b-dca4a41dbf5a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.313121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1011/1992] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:01 2022] PUT /identity/v3/projects/895a56a4fb9e48cd8635792dcf3d4cf9/users/26fee53a0fc34a70be9eef9d9f117f57/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:01.323865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc892a-1f76-4e4e-94a7-0bba7251b190 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.324154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc892a-1f76-4e4e-94a7-0bba7251b190 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.324432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc892a-1f76-4e4e-94a7-0bba7251b190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.350590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.352088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.356683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-35cc892a-1f76-4e4e-94a7-0bba7251b190 None None] MFA Rules not processed for user `26fee53a0fc34a70be9eef9d9f117f57`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:01.357956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-35cc892a-1f76-4e4e-94a7-0bba7251b190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:01.362786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 982/1993] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:01.436219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1012/1994] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:01.447605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.456884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54138ee-d5f0-46cb-92dc-b5ae1260edfb service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.506775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982106d4-6afd-4252-8b97-6b51c08314a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.507303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982106d4-6afd-4252-8b97-6b51c08314a9 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.507581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982106d4-6afd-4252-8b97-6b51c08314a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.507851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982106d4-6afd-4252-8b97-6b51c08314a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.516075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982106d4-6afd-4252-8b97-6b51c08314a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.517578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982106d4-6afd-4252-8b97-6b51c08314a9 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.526865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 983/1995] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:01.550914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc892a-1f76-4e4e-94a7-0bba7251b190 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.599157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:01.628265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc892a-1f76-4e4e-94a7-0bba7251b190 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.633643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.634378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.634791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.635129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] PATH_INFO: `/v3/users/26fee53a0fc34a70be9eef9d9f117f57` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.636385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26fee53a0fc34a70be9eef9d9f117f57)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.638567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.695250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26fee53a0fc34a70be9eef9d9f117f57'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:01.702885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26fee53a0fc34a70be9eef9d9f117f57'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:01.703556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26fee53a0fc34a70be9eef9d9f117f57'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:01.717512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26fee53a0fc34a70be9eef9d9f117f57'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:01.725357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26fee53a0fc34a70be9eef9d9f117f57'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:01.732032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1013/1996] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:01 2022] DELETE /identity/v3/users/26fee53a0fc34a70be9eef9d9f117f57 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:01.736480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec814fb4-d123-4c67-8ad9-9f2a1af8cbd9 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.736894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec814fb4-d123-4c67-8ad9-9f2a1af8cbd9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.737069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec814fb4-d123-4c67-8ad9-9f2a1af8cbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.756957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ec814fb4-d123-4c67-8ad9-9f2a1af8cbd9 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:01.758140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ec814fb4-d123-4c67-8ad9-9f2a1af8cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:01.818460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:01.833487 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 984/1997] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:01.843082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:01.844771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:01.923195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d41e9c-97d8-44ce-beea-e9927ee5b0d4 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:01.927868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c84efe-3874-4e2f-be2c-12c3efdc014d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:01.928541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c84efe-3874-4e2f-be2c-12c3efdc014d service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:01.928817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c84efe-3874-4e2f-be2c-12c3efdc014d service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:01.929112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c84efe-3874-4e2f-be2c-12c3efdc014d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:01.938639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c84efe-3874-4e2f-be2c-12c3efdc014d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:01.940069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c84efe-3874-4e2f-be2c-12c3efdc014d service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:01.949724 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1014/1998] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:02.213544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec814fb4-d123-4c67-8ad9-9f2a1af8cbd9 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:02.215015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ec814fb4-d123-4c67-8ad9-9f2a1af8cbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:02.228396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec814fb4-d123-4c67-8ad9-9f2a1af8cbd9 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:02.233571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:02.234286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.234569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.234845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] PATH_INFO: `/v3/projects/895a56a4fb9e48cd8635792dcf3d4cf9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.236111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=895a56a4fb9e48cd8635792dcf3d4cf9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:02.238166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:02.286311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '895a56a4fb9e48cd8635792dcf3d4cf9', 'request_id': 'req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:02.291903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '895a56a4fb9e48cd8635792dcf3d4cf9', 'request_id': 'req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:02.298155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '895a56a4fb9e48cd8635792dcf3d4cf9', 'request_id': 'req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:02.310604 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] The token cache is being invalidate because project 895a56a4fb9e48cd8635792dcf3d4cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:02.310890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:02.311525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7cb067fc-5c7d-4e11-b3fb-377431a77cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:02.313019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 985/1999] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:02 2022] DELETE /identity/v3/projects/895a56a4fb9e48cd8635792dcf3d4cf9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:02.317957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb459350-1c26-4ac8-840d-ba9123a7e67b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.318244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb459350-1c26-4ac8-840d-ba9123a7e67b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.318518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb459350-1c26-4ac8-840d-ba9123a7e67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.337585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bb459350-1c26-4ac8-840d-ba9123a7e67b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:02.338809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bb459350-1c26-4ac8-840d-ba9123a7e67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:02.396255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1015/2000] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:02.405595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3bbf8-a0fa-408c-9a55-80d3d837a481 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.406013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3bbf8-a0fa-408c-9a55-80d3d837a481 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.406353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3bbf8-a0fa-408c-9a55-80d3d837a481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.430496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1ad3bbf8-a0fa-408c-9a55-80d3d837a481 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:02.432656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1ad3bbf8-a0fa-408c-9a55-80d3d837a481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:02.441079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 986/2001] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:02.445383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb459350-1c26-4ac8-840d-ba9123a7e67b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:02.452331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:02.455994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb459350-1c26-4ac8-840d-ba9123a7e67b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:02.462175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:02.462975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.463609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.463966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.464759 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:02.465147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:02.467494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:02.487156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1016/2002] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:02.488861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3bbf8-a0fa-408c-9a55-80d3d837a481 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:02.500784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3bbf8-a0fa-408c-9a55-80d3d837a481 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:02.507464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70338cea-3eb9-4794-8707-df9e17cafdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:02.508093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70338cea-3eb9-4794-8707-df9e17cafdbf admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.508492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70338cea-3eb9-4794-8707-df9e17cafdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.508654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70338cea-3eb9-4794-8707-df9e17cafdbf admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.509296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70338cea-3eb9-4794-8707-df9e17cafdbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:02.511281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70338cea-3eb9-4794-8707-df9e17cafdbf admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:02.547572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 987/2003] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:02 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:02.553008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:02.571412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beecc0d-7b7c-4d87-b433-45b26675ec4a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:02.635678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f160688-9441-4926-a0db-e18e52ae47a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:02.636295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f160688-9441-4926-a0db-e18e52ae47a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.636578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f160688-9441-4926-a0db-e18e52ae47a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.636852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f160688-9441-4926-a0db-e18e52ae47a6 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.637484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f160688-9441-4926-a0db-e18e52ae47a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:02.639442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f160688-9441-4926-a0db-e18e52ae47a6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:02.663934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1017/2004] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:02 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:02.668107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70338cea-3eb9-4794-8707-df9e17cafdbf admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:02.680494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70338cea-3eb9-4794-8707-df9e17cafdbf admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:02.685399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d6f9d-f354-4f4e-aa67-ff3ad2521c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:02.686109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d6f9d-f354-4f4e-aa67-ff3ad2521c53 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.686472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d6f9d-f354-4f4e-aa67-ff3ad2521c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.686816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d6f9d-f354-4f4e-aa67-ff3ad2521c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.687537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d6f9d-f354-4f4e-aa67-ff3ad2521c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:02.689463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d6f9d-f354-4f4e-aa67-ff3ad2521c53 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:02.702583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 988/2005] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:02.707147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f160688-9441-4926-a0db-e18e52ae47a6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:02.718037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f160688-9441-4926-a0db-e18e52ae47a6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:02.722869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05260324-8f25-4cec-b345-7f577a9c36f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:02.723630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260324-8f25-4cec-b345-7f577a9c36f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.724016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260324-8f25-4cec-b345-7f577a9c36f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.724336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05260324-8f25-4cec-b345-7f577a9c36f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.725047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05260324-8f25-4cec-b345-7f577a9c36f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:02.739528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05260324-8f25-4cec-b345-7f577a9c36f2 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:02.751768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1018/2006] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:02.756460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d6f9d-f354-4f4e-aa67-ff3ad2521c53 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:02.766806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d6f9d-f354-4f4e-aa67-ff3ad2521c53 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:02.773224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7855c5-f599-40c2-bc7d-397fa937c371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:02.773930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7855c5-f599-40c2-bc7d-397fa937c371 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.774381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7855c5-f599-40c2-bc7d-397fa937c371 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.774656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7855c5-f599-40c2-bc7d-397fa937c371 admin admin] PATH_INFO: `/v3/projects/c5924a5fef0b4f75825f3ec03cb6999a/users/5b629c0a846d4cb3b390161f9581d133/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.803152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7855c5-f599-40c2-bc7d-397fa937c371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5924a5fef0b4f75825f3ec03cb6999a, user_id=5b629c0a846d4cb3b390161f9581d133, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:02.806057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7855c5-f599-40c2-bc7d-397fa937c371 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:02.819757 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 989/2007] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:02 2022] PUT /identity/v3/projects/c5924a5fef0b4f75825f3ec03cb6999a/users/5b629c0a846d4cb3b390161f9581d133/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:02.829961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d02c4-a2f1-4e99-a557-e148b57de6d3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:02.830248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d02c4-a2f1-4e99-a557-e148b57de6d3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:02.830541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d02c4-a2f1-4e99-a557-e148b57de6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:02.860049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4d0d02c4-a2f1-4e99-a557-e148b57de6d3 None None] MFA Rules not processed for user `5b629c0a846d4cb3b390161f9581d133`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:02.861261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4d0d02c4-a2f1-4e99-a557-e148b57de6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:02.933019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1019/2008] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:02.937512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7855c5-f599-40c2-bc7d-397fa937c371 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.005389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7855c5-f599-40c2-bc7d-397fa937c371 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.015385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1b01b-53b3-47b3-80e7-245d2f97052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.015385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1b01b-53b3-47b3-80e7-245d2f97052e admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.015385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1b01b-53b3-47b3-80e7-245d2f97052e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.015385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e1b01b-53b3-47b3-80e7-245d2f97052e admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.015803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1b01b-53b3-47b3-80e7-245d2f97052e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.015803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e1b01b-53b3-47b3-80e7-245d2f97052e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.038208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:03.048120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 990/2009] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:02 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:03.049806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d02c4-a2f1-4e99-a557-e148b57de6d3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.061025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d02c4-a2f1-4e99-a557-e148b57de6d3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.119612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4edf-e5b8-4241-8c1b-5d756511e710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.120223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4edf-e5b8-4241-8c1b-5d756511e710 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.120499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4edf-e5b8-4241-8c1b-5d756511e710 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.120770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a4edf-e5b8-4241-8c1b-5d756511e710 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.121393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a4edf-e5b8-4241-8c1b-5d756511e710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.123351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a4edf-e5b8-4241-8c1b-5d756511e710 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.131253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:03.145306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1020/2010] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:03.149269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1b01b-53b3-47b3-80e7-245d2f97052e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.159459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e1b01b-53b3-47b3-80e7-245d2f97052e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.164493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d1ffa-055f-4420-8b92-dd7a359217c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.165086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d1ffa-055f-4420-8b92-dd7a359217c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.165364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d1ffa-055f-4420-8b92-dd7a359217c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.165661 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d1ffa-055f-4420-8b92-dd7a359217c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.166337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d1ffa-055f-4420-8b92-dd7a359217c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.167917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d1ffa-055f-4420-8b92-dd7a359217c9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.179894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 991/2011] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:03.183842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4edf-e5b8-4241-8c1b-5d756511e710 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.195264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a4edf-e5b8-4241-8c1b-5d756511e710 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.199914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3c385-64fd-4a14-a73a-3e9e26332b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.200516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c385-64fd-4a14-a73a-3e9e26332b2a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.200794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c385-64fd-4a14-a73a-3e9e26332b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.201124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c385-64fd-4a14-a73a-3e9e26332b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.201812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c385-64fd-4a14-a73a-3e9e26332b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.203390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c385-64fd-4a14-a73a-3e9e26332b2a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.213362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1021/2012] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:03.217442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d1ffa-055f-4420-8b92-dd7a359217c9 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.230045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d1ffa-055f-4420-8b92-dd7a359217c9 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.234590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb197dce-3e8d-444e-ab41-d801512e369d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.235241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197dce-3e8d-444e-ab41-d801512e369d admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.235518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197dce-3e8d-444e-ab41-d801512e369d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.235787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197dce-3e8d-444e-ab41-d801512e369d admin admin] PATH_INFO: `/v3/projects/213aec7078ba468e95ef4f441af43730/users/ae6431e791c34df98c131c78cc60c3b5/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.250692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb197dce-3e8d-444e-ab41-d801512e369d admin admin] RBAC: Authorizing `identity:create_grant(project_id=213aec7078ba468e95ef4f441af43730, user_id=ae6431e791c34df98c131c78cc60c3b5, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.253345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb197dce-3e8d-444e-ab41-d801512e369d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.266681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 992/2013] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:03 2022] PUT /identity/v3/projects/213aec7078ba468e95ef4f441af43730/users/ae6431e791c34df98c131c78cc60c3b5/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:03.277347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ee69-0eec-40e8-8eb0-c6e0fa3b8615 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.277656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ee69-0eec-40e8-8eb0-c6e0fa3b8615 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.277932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ee69-0eec-40e8-8eb0-c6e0fa3b8615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.306044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9004ee69-0eec-40e8-8eb0-c6e0fa3b8615 None None] MFA Rules not processed for user `ae6431e791c34df98c131c78cc60c3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:03.307166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9004ee69-0eec-40e8-8eb0-c6e0fa3b8615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:03.369927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1022/2014] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:03.374594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb197dce-3e8d-444e-ab41-d801512e369d admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.435468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb197dce-3e8d-444e-ab41-d801512e369d admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.440288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80a1c3-72e9-44ab-bb6c-c23afb40beb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.440894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a1c3-72e9-44ab-bb6c-c23afb40beb7 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.441205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a1c3-72e9-44ab-bb6c-c23afb40beb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.442460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80a1c3-72e9-44ab-bb6c-c23afb40beb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.442639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80a1c3-72e9-44ab-bb6c-c23afb40beb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.444753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80a1c3-72e9-44ab-bb6c-c23afb40beb7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.470766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 993/2015] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:03.474952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004ee69-0eec-40e8-8eb0-c6e0fa3b8615 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.485490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004ee69-0eec-40e8-8eb0-c6e0fa3b8615 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.540398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f951d7-7c22-4e1e-bc97-e6ec1ab674d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.541079 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f951d7-7c22-4e1e-bc97-e6ec1ab674d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.541364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f951d7-7c22-4e1e-bc97-e6ec1ab674d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.541695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f951d7-7c22-4e1e-bc97-e6ec1ab674d2 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.542462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f951d7-7c22-4e1e-bc97-e6ec1ab674d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.544803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f951d7-7c22-4e1e-bc97-e6ec1ab674d2 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.566527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1023/2016] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:03.571211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80a1c3-72e9-44ab-bb6c-c23afb40beb7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.581740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80a1c3-72e9-44ab-bb6c-c23afb40beb7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.586356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3aa3-d02d-40a8-8dfb-25e66a6c2880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.586950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3aa3-d02d-40a8-8dfb-25e66a6c2880 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.587233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3aa3-d02d-40a8-8dfb-25e66a6c2880 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.587584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a3aa3-d02d-40a8-8dfb-25e66a6c2880 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.588225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a3aa3-d02d-40a8-8dfb-25e66a6c2880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.589865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a3aa3-d02d-40a8-8dfb-25e66a6c2880 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.603820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 994/2017] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:03.605437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f951d7-7c22-4e1e-bc97-e6ec1ab674d2 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.615752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f951d7-7c22-4e1e-bc97-e6ec1ab674d2 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.620473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8920a9-ac79-4a97-829b-a89a86974c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.621138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8920a9-ac79-4a97-829b-a89a86974c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.621418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8920a9-ac79-4a97-829b-a89a86974c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.621727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8920a9-ac79-4a97-829b-a89a86974c69 admin admin] PATH_INFO: `/v3/projects/88743c8e943c49f2891845179f9e250e/users/cccacd7a0f874a86b514f74654126493/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.638146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8920a9-ac79-4a97-829b-a89a86974c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88743c8e943c49f2891845179f9e250e, user_id=cccacd7a0f874a86b514f74654126493, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.640794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8920a9-ac79-4a97-829b-a89a86974c69 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.655878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1024/2018] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:03 2022] PUT /identity/v3/projects/88743c8e943c49f2891845179f9e250e/users/cccacd7a0f874a86b514f74654126493/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:03.659955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3aa3-d02d-40a8-8dfb-25e66a6c2880 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.671273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a3aa3-d02d-40a8-8dfb-25e66a6c2880 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.729737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a8287-c5c0-4c1f-9a6b-c035fceaa09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.730348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8287-c5c0-4c1f-9a6b-c035fceaa09a admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.730628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8287-c5c0-4c1f-9a6b-c035fceaa09a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.730901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a8287-c5c0-4c1f-9a6b-c035fceaa09a admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.731535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8287-c5c0-4c1f-9a6b-c035fceaa09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.733143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a8287-c5c0-4c1f-9a6b-c035fceaa09a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.743645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 995/2019] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:03.747888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8920a9-ac79-4a97-829b-a89a86974c69 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.767435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8920a9-ac79-4a97-829b-a89a86974c69 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.772184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.772835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.773145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.773409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] PATH_INFO: `/v3/projects/88743c8e943c49f2891845179f9e250e/users/cccacd7a0f874a86b514f74654126493/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.775342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88743c8e943c49f2891845179f9e250e, user_id=cccacd7a0f874a86b514f74654126493, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.778007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.788146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1025/2020] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:03 2022] PUT /identity/v3/projects/88743c8e943c49f2891845179f9e250e/users/cccacd7a0f874a86b514f74654126493/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:03.858159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a91d0-1f8f-457c-9114-e2a000626b83 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.858736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a91d0-1f8f-457c-9114-e2a000626b83 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.858996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a91d0-1f8f-457c-9114-e2a000626b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.886836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ed7a91d0-1f8f-457c-9114-e2a000626b83 None None] MFA Rules not processed for user `cccacd7a0f874a86b514f74654126493`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:03.888032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ed7a91d0-1f8f-457c-9114-e2a000626b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:03.911305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:03.957558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 996/2021] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:03.968618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.970305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:03.994862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:04.031324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f1d4a-2c69-4db5-9ab7-6270cd905240 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.034985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9798ae3-1ab3-4a64-8159-398122dfb3e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.035612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9798ae3-1ab3-4a64-8159-398122dfb3e6 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.035888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9798ae3-1ab3-4a64-8159-398122dfb3e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.036159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9798ae3-1ab3-4a64-8159-398122dfb3e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.091646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9798ae3-1ab3-4a64-8159-398122dfb3e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.093001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9798ae3-1ab3-4a64-8159-398122dfb3e6 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.102291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1026/2022] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:10.910643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a91d0-1f8f-457c-9114-e2a000626b83 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:10.921113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a91d0-1f8f-457c-9114-e2a000626b83 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:10.927000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:10.927685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:10.927968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:10.928239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:10.996832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:10.998565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:11.008776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 997/2023] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:11.602754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9798ae3-1ab3-4a64-8159-398122dfb3e6 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:11.621275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9798ae3-1ab3-4a64-8159-398122dfb3e6 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:11.625552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c47d7-01dc-41a1-96fc-ff82f70797e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:11.626528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c47d7-01dc-41a1-96fc-ff82f70797e5 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:11.626993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c47d7-01dc-41a1-96fc-ff82f70797e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:11.627469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c47d7-01dc-41a1-96fc-ff82f70797e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:11.642179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c47d7-01dc-41a1-96fc-ff82f70797e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:11.642179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c47d7-01dc-41a1-96fc-ff82f70797e5 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:11.652752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1027/2024] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:11.691130 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:11.693547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:11.791744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d94a3-2931-4f71-82c7-8b0a2f19f60d service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:11.797235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:11.798291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:11.798651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:11.799410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] PATH_INFO: `/v3/users/47ec3a8bf2fa487a9304745003e428d9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:11.800590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ec3a8bf2fa487a9304745003e428d9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:11.803050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:11.852636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ec3a8bf2fa487a9304745003e428d9'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:11.860505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ec3a8bf2fa487a9304745003e428d9'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:11.861159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ec3a8bf2fa487a9304745003e428d9'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:11.867014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:11.877719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ec3a8bf2fa487a9304745003e428d9'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:11.885221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ec3a8bf2fa487a9304745003e428d9'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:11.892127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 998/2025] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:11 2022] DELETE /identity/v3/users/47ec3a8bf2fa487a9304745003e428d9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:11.897206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5d36-f2a1-4d38-a5ea-9a4322f7485b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:11.897496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5d36-f2a1-4d38-a5ea-9a4322f7485b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:11.897815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e5d36-f2a1-4d38-a5ea-9a4322f7485b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:11.917774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d02e5d36-f2a1-4d38-a5ea-9a4322f7485b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:11.919043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d02e5d36-f2a1-4d38-a5ea-9a4322f7485b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:11.978797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1028/2026] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:11.989679 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:11.991202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:12.063802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f88e8-1b0a-411e-a2fa-32aaa9d7c57a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:12.067479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f910ee-02c3-4fc3-911b-61929126aadf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:12.068094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f910ee-02c3-4fc3-911b-61929126aadf service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:12.068378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f910ee-02c3-4fc3-911b-61929126aadf service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:12.068660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f910ee-02c3-4fc3-911b-61929126aadf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:12.078214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f910ee-02c3-4fc3-911b-61929126aadf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:12.080163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f910ee-02c3-4fc3-911b-61929126aadf service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:12.090011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 999/2027] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:12.420979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e5d36-f2a1-4d38-a5ea-9a4322f7485b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:12.422743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d02e5d36-f2a1-4d38-a5ea-9a4322f7485b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:12.435485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e5d36-f2a1-4d38-a5ea-9a4322f7485b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:12.440520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:12.441162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:12.441992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:12.442263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] PATH_INFO: `/v3/projects/4a31ab11fa4c442d93a2075a29d05a4d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:12.443417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a31ab11fa4c442d93a2075a29d05a4d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:12.445366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:12.500427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a31ab11fa4c442d93a2075a29d05a4d', 'request_id': 'req-cac3b504-d305-4f75-9596-3ebba7518731'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.506000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:12.507476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a31ab11fa4c442d93a2075a29d05a4d', 'request_id': 'req-cac3b504-d305-4f75-9596-3ebba7518731'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.514787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a31ab11fa4c442d93a2075a29d05a4d', 'request_id': 'req-cac3b504-d305-4f75-9596-3ebba7518731'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.527850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] The token cache is being invalidate because project 4a31ab11fa4c442d93a2075a29d05a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:12.528144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.528769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.530338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1029/2028] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:12 2022] DELETE /identity/v3/projects/4a31ab11fa4c442d93a2075a29d05a4d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:12.534537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f910ee-02c3-4fc3-911b-61929126aadf service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:12.544962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f910ee-02c3-4fc3-911b-61929126aadf service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:12.607214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:12.607857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:12.608139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:12.608414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] PATH_INFO: `/v3/users/30ba672ac28d423d9ce4c92c4bb7a7d8` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:12.609532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ba672ac28d423d9ce4c92c4bb7a7d8)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:12.611529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:12.672078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ba672ac28d423d9ce4c92c4bb7a7d8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.683313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ba672ac28d423d9ce4c92c4bb7a7d8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.690820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ba672ac28d423d9ce4c92c4bb7a7d8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.705572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ba672ac28d423d9ce4c92c4bb7a7d8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.713635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ba672ac28d423d9ce4c92c4bb7a7d8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:12.716135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:12.721227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1000/2029] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:12 2022] DELETE /identity/v3/users/30ba672ac28d423d9ce4c92c4bb7a7d8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:13.024929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.026576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.044419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b504-d305-4f75-9596-3ebba7518731 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.120487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.121382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.121758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.122098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] PATH_INFO: `/v3/projects/881e89929e274428992c863800b066e2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.123290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=881e89929e274428992c863800b066e2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.125324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.186280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '881e89929e274428992c863800b066e2', 'request_id': 'req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.192014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '881e89929e274428992c863800b066e2', 'request_id': 'req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.198392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '881e89929e274428992c863800b066e2', 'request_id': 'req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.211408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] The token cache is being invalidate because project 881e89929e274428992c863800b066e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:13.211694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.212332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.213967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1030/2030] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:13 2022] DELETE /identity/v3/projects/881e89929e274428992c863800b066e2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:13.218180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.219608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.232558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f50147-a7a4-4b71-bdd3-a739ce9c067e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.291072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.291711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.291989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.292260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] PATH_INFO: `/v3/users/aa7fd212c31341ce926ee60ff1451e0a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.293387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7fd212c31341ce926ee60ff1451e0a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.295349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.344545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7fd212c31341ce926ee60ff1451e0a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.352599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7fd212c31341ce926ee60ff1451e0a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.353417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7fd212c31341ce926ee60ff1451e0a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.366736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7fd212c31341ce926ee60ff1451e0a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.374037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-85b129ce-2b29-4d82-a9d0-89ec038288f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7fd212c31341ce926ee60ff1451e0a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.380531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1001/2031] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:13 2022] DELETE /identity/v3/users/aa7fd212c31341ce926ee60ff1451e0a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:13.645345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.646936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.660105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d1e7d-baf8-40b5-a083-93eaa30544c6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.719995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.720636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.720911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.721183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] PATH_INFO: `/v3/projects/2c6e632dc08d40469fbdee90a93efcfd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.722336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6e632dc08d40469fbdee90a93efcfd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.724372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.741555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:13.785519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6e632dc08d40469fbdee90a93efcfd', 'request_id': 'req-14b85c67-50ae-459a-933d-d78b600242f4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.794961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6e632dc08d40469fbdee90a93efcfd', 'request_id': 'req-14b85c67-50ae-459a-933d-d78b600242f4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.801141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6e632dc08d40469fbdee90a93efcfd', 'request_id': 'req-14b85c67-50ae-459a-933d-d78b600242f4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.821291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] The token cache is being invalidate because project 2c6e632dc08d40469fbdee90a93efcfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:13.821593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.822398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-14b85c67-50ae-459a-933d-d78b600242f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:13.827367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1031/2032] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:13 2022] DELETE /identity/v3/projects/2c6e632dc08d40469fbdee90a93efcfd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:13.830412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffebef2-8a9c-400c-8d6a-199eba8d8435 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.830806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffebef2-8a9c-400c-8d6a-199eba8d8435 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.831148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffebef2-8a9c-400c-8d6a-199eba8d8435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.855973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-bffebef2-8a9c-400c-8d6a-199eba8d8435 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.857228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bffebef2-8a9c-400c-8d6a-199eba8d8435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.919932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1002/2033] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.928629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff81c01-7fb2-43e7-a6fa-806a1fa89aee None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.928920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff81c01-7fb2-43e7-a6fa-806a1fa89aee None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.929204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff81c01-7fb2-43e7-a6fa-806a1fa89aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.950589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-aff81c01-7fb2-43e7-a6fa-806a1fa89aee None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.951824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-aff81c01-7fb2-43e7-a6fa-806a1fa89aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.961099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1032/2034] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.965167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffebef2-8a9c-400c-8d6a-199eba8d8435 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.976254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffebef2-8a9c-400c-8d6a-199eba8d8435 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.980990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.981567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.981882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.982348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.982864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:13.983245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.984898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.001137 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1003/2035] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:14.002954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff81c01-7fb2-43e7-a6fa-806a1fa89aee None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.019894 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff81c01-7fb2-43e7-a6fa-806a1fa89aee None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.025981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed41b51-855e-443a-994a-0511f018213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.026813 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed41b51-855e-443a-994a-0511f018213e admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.027113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed41b51-855e-443a-994a-0511f018213e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.027435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed41b51-855e-443a-994a-0511f018213e admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.028232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed41b51-855e-443a-994a-0511f018213e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.030753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed41b51-855e-443a-994a-0511f018213e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.064700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1033/2036] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.069197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.082479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3e49a-dd42-4f45-b02b-cb2abd788a7f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.086402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.147046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd5ff8-731e-40d0-8c54-8e0a46700fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.147654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd5ff8-731e-40d0-8c54-8e0a46700fcd admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.147932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd5ff8-731e-40d0-8c54-8e0a46700fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.148201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd5ff8-731e-40d0-8c54-8e0a46700fcd admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.148837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd5ff8-731e-40d0-8c54-8e0a46700fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.150818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd5ff8-731e-40d0-8c54-8e0a46700fcd admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.174479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1004/2037] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:14.178660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed41b51-855e-443a-994a-0511f018213e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.189353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed41b51-855e-443a-994a-0511f018213e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.194095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33d02d-e41a-47d4-a077-22dd828a43b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.194698 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33d02d-e41a-47d4-a077-22dd828a43b0 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.194975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33d02d-e41a-47d4-a077-22dd828a43b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.195246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33d02d-e41a-47d4-a077-22dd828a43b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.195888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33d02d-e41a-47d4-a077-22dd828a43b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.197740 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc33d02d-e41a-47d4-a077-22dd828a43b0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.208320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1034/2038] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.212837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd5ff8-731e-40d0-8c54-8e0a46700fcd admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.225486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd5ff8-731e-40d0-8c54-8e0a46700fcd admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.230876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ca07b-300b-4176-83c8-86cd193247a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.231643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ca07b-300b-4176-83c8-86cd193247a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.231928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ca07b-300b-4176-83c8-86cd193247a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.232201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ca07b-300b-4176-83c8-86cd193247a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.232845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ca07b-300b-4176-83c8-86cd193247a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.234481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6ca07b-300b-4176-83c8-86cd193247a9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.246639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1005/2039] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.251027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33d02d-e41a-47d4-a077-22dd828a43b0 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.263235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc33d02d-e41a-47d4-a077-22dd828a43b0 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.268198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add2169-65da-4f69-a3d5-2f9cdd527cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.268856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add2169-65da-4f69-a3d5-2f9cdd527cfd admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.269133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add2169-65da-4f69-a3d5-2f9cdd527cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.269593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add2169-65da-4f69-a3d5-2f9cdd527cfd admin admin] PATH_INFO: `/v3/projects/dd7c0f339f2b436a80010b06132e6822/users/3f887898ce93492d80a2ed658d9b8641/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.285399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add2169-65da-4f69-a3d5-2f9cdd527cfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7c0f339f2b436a80010b06132e6822, user_id=3f887898ce93492d80a2ed658d9b8641, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.288163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add2169-65da-4f69-a3d5-2f9cdd527cfd admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.307159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1035/2040] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:14 2022] PUT /identity/v3/projects/dd7c0f339f2b436a80010b06132e6822/users/3f887898ce93492d80a2ed658d9b8641/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:14.323369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a988b47-7a46-442d-9001-2be0b462955a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.323906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a988b47-7a46-442d-9001-2be0b462955a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.324416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a988b47-7a46-442d-9001-2be0b462955a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.373112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1a988b47-7a46-442d-9001-2be0b462955a None None] MFA Rules not processed for user `3f887898ce93492d80a2ed658d9b8641`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:14.374764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1a988b47-7a46-442d-9001-2be0b462955a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:14.458635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1006/2041] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:14.463713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add2169-65da-4f69-a3d5-2f9cdd527cfd admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.558653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add2169-65da-4f69-a3d5-2f9cdd527cfd admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.565205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413fa40-aee9-42e2-a22a-26b1e19be50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.565881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413fa40-aee9-42e2-a22a-26b1e19be50f admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.566224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413fa40-aee9-42e2-a22a-26b1e19be50f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.566563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413fa40-aee9-42e2-a22a-26b1e19be50f admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.567422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413fa40-aee9-42e2-a22a-26b1e19be50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.570128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413fa40-aee9-42e2-a22a-26b1e19be50f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.603546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1036/2042] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.607652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a988b47-7a46-442d-9001-2be0b462955a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.637196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a988b47-7a46-442d-9001-2be0b462955a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.701856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc09e67-88cf-4fe1-b9bb-ee979483d5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.702482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc09e67-88cf-4fe1-b9bb-ee979483d5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.702766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc09e67-88cf-4fe1-b9bb-ee979483d5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.703041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc09e67-88cf-4fe1-b9bb-ee979483d5c3 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.703693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc09e67-88cf-4fe1-b9bb-ee979483d5c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.706096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc09e67-88cf-4fe1-b9bb-ee979483d5c3 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.729525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1007/2043] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:14.733634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413fa40-aee9-42e2-a22a-26b1e19be50f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.746065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413fa40-aee9-42e2-a22a-26b1e19be50f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.750847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13223b-f567-499a-85c6-7f6d8d773a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.751452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13223b-f567-499a-85c6-7f6d8d773a0d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.751735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13223b-f567-499a-85c6-7f6d8d773a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.752036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13223b-f567-499a-85c6-7f6d8d773a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.752686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13223b-f567-499a-85c6-7f6d8d773a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.754433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13223b-f567-499a-85c6-7f6d8d773a0d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.766107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1037/2044] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:14.770736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc09e67-88cf-4fe1-b9bb-ee979483d5c3 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.778193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.782657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc09e67-88cf-4fe1-b9bb-ee979483d5c3 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.794364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.794364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.794364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.794364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.795001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.796649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.811168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1008/2045] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.816433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13223b-f567-499a-85c6-7f6d8d773a0d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.832395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13223b-f567-499a-85c6-7f6d8d773a0d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.840688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b222a-1f1e-4255-8c5e-6a67ed51f5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.841885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b222a-1f1e-4255-8c5e-6a67ed51f5c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.842167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b222a-1f1e-4255-8c5e-6a67ed51f5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.842445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b222a-1f1e-4255-8c5e-6a67ed51f5c9 admin admin] PATH_INFO: `/v3/projects/d58a87728da34b67a63454017e7dc3fd/users/1b8ea963087b4536862f39e0f1c8ad49/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.857578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.862248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b222a-1f1e-4255-8c5e-6a67ed51f5c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58a87728da34b67a63454017e7dc3fd, user_id=1b8ea963087b4536862f39e0f1c8ad49, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.866303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b222a-1f1e-4255-8c5e-6a67ed51f5c9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.868895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.872085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:14.882980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1038/2046] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:14 2022] PUT /identity/v3/projects/d58a87728da34b67a63454017e7dc3fd/users/1b8ea963087b4536862f39e0f1c8ad49/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:14.889760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d357bab-a187-494b-aa1b-17ee494a6f87 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.893100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ed27-b1ab-448a-87e1-528f5900c5e3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.893457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ed27-b1ab-448a-87e1-528f5900c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.893764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17ed27-b1ab-448a-87e1-528f5900c5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.925023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-0c17ed27-b1ab-448a-87e1-528f5900c5e3 None None] MFA Rules not processed for user `1b8ea963087b4536862f39e0f1c8ad49`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:14.926437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0c17ed27-b1ab-448a-87e1-528f5900c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:14.955020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.955654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.955965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.956249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] PATH_INFO: `/v3/users/5b7bacd3026045759540b1b7dd10c8dd` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.957989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7bacd3026045759540b1b7dd10c8dd)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.959920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.001345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1039/2047] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:15.007814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17ed27-b1ab-448a-87e1-528f5900c5e3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.017767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7bacd3026045759540b1b7dd10c8dd'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:15.018689 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17ed27-b1ab-448a-87e1-528f5900c5e3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.025048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.026014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.026542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.026542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.026702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7bacd3026045759540b1b7dd10c8dd'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:15.026930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.027227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7bacd3026045759540b1b7dd10c8dd'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:15.028876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.041580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7bacd3026045759540b1b7dd10c8dd'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:15.049980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-2b3bbe35-1d33-4863-b9a6-ac41e66a1a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7bacd3026045759540b1b7dd10c8dd'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:15.057043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1009/2048] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:14 2022] DELETE /identity/v3/users/5b7bacd3026045759540b1b7dd10c8dd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:15.058246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1040/2049] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:15.061298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42f5c1-7619-4f45-a65f-48dc0ad56e0a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.061585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42f5c1-7619-4f45-a65f-48dc0ad56e0a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.061890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42f5c1-7619-4f45-a65f-48dc0ad56e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.062065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.063670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.081386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-af42f5c1-7619-4f45-a65f-48dc0ad56e0a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.083185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-af42f5c1-7619-4f45-a65f-48dc0ad56e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.130280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d08fe-3fac-45bd-917c-270a8685f505 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.135165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd8ef4-1993-47d9-8520-e4c9ab47aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.136071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd8ef4-1993-47d9-8520-e4c9ab47aae6 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.136356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd8ef4-1993-47d9-8520-e4c9ab47aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.136982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dd8ef4-1993-47d9-8520-e4c9ab47aae6 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.138114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd8ef4-1993-47d9-8520-e4c9ab47aae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.140145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dd8ef4-1993-47d9-8520-e4c9ab47aae6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.148778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1010/2050] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.156203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42f5c1-7619-4f45-a65f-48dc0ad56e0a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.157893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-af42f5c1-7619-4f45-a65f-48dc0ad56e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.167710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1041/2051] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:15.170655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd8ef4-1993-47d9-8520-e4c9ab47aae6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.178423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:15.185295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dd8ef4-1993-47d9-8520-e4c9ab47aae6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.191032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6d541-e3cf-4125-bf6d-e7024d69c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.191466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6d541-e3cf-4125-bf6d-e7024d69c245 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.191749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6d541-e3cf-4125-bf6d-e7024d69c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.192028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c6d541-e3cf-4125-bf6d-e7024d69c245 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.192679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6d541-e3cf-4125-bf6d-e7024d69c245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.194338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c6d541-e3cf-4125-bf6d-e7024d69c245 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.223217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1042/2052] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:15.224730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6d541-e3cf-4125-bf6d-e7024d69c245 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.230197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42f5c1-7619-4f45-a65f-48dc0ad56e0a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.235173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b9c78-a8c2-48f6-b1ee-3ed8ccb59090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.235750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c6d541-e3cf-4125-bf6d-e7024d69c245 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.235859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b9c78-a8c2-48f6-b1ee-3ed8ccb59090 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.236074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b9c78-a8c2-48f6-b1ee-3ed8ccb59090 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.236398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b9c78-a8c2-48f6-b1ee-3ed8ccb59090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.248280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5d7eb-b4ae-460d-a8ab-839f2176c90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.249704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b9c78-a8c2-48f6-b1ee-3ed8ccb59090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.250074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d7eb-b4ae-460d-a8ab-839f2176c90c admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.250074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d7eb-b4ae-460d-a8ab-839f2176c90c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.250074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d7eb-b4ae-460d-a8ab-839f2176c90c admin admin] PATH_INFO: `/v3/projects/a281a2676a3045a9a4da9ad7e03032d0/users/ff69128d173d4af784487e31b93b5746/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.250303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b9c78-a8c2-48f6-b1ee-3ed8ccb59090 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.265568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5d7eb-b4ae-460d-a8ab-839f2176c90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a281a2676a3045a9a4da9ad7e03032d0, user_id=ff69128d173d4af784487e31b93b5746, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.266248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1011/2053] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:15.268538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5d7eb-b4ae-460d-a8ab-839f2176c90c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.287087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1043/2054] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:15 2022] PUT /identity/v3/projects/a281a2676a3045a9a4da9ad7e03032d0/users/ff69128d173d4af784487e31b93b5746/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:15.291243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b9c78-a8c2-48f6-b1ee-3ed8ccb59090 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.303056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b9c78-a8c2-48f6-b1ee-3ed8ccb59090 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.370139 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3921c-337a-4b07-8b18-39044f92ef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.370761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3921c-337a-4b07-8b18-39044f92ef7c admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.371046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3921c-337a-4b07-8b18-39044f92ef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.371368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3921c-337a-4b07-8b18-39044f92ef7c admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.372032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3921c-337a-4b07-8b18-39044f92ef7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.373915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3921c-337a-4b07-8b18-39044f92ef7c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.387910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1012/2055] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:15.392258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5d7eb-b4ae-460d-a8ab-839f2176c90c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.404345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5d7eb-b4ae-460d-a8ab-839f2176c90c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.409356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.410658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.411067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.411264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] PATH_INFO: `/v3/projects/a281a2676a3045a9a4da9ad7e03032d0/users/ff69128d173d4af784487e31b93b5746/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.413277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a281a2676a3045a9a4da9ad7e03032d0, user_id=ff69128d173d4af784487e31b93b5746, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.421124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.428207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:15.433093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1044/2056] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:15 2022] PUT /identity/v3/projects/a281a2676a3045a9a4da9ad7e03032d0/users/ff69128d173d4af784487e31b93b5746/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:15.444764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ef0a2-b765-468d-8bcc-208811f59b37 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.445061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ef0a2-b765-468d-8bcc-208811f59b37 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.445430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ef0a2-b765-468d-8bcc-208811f59b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.478819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-6f9ef0a2-b765-468d-8bcc-208811f59b37 None None] MFA Rules not processed for user `ff69128d173d4af784487e31b93b5746`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.480060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6f9ef0a2-b765-468d-8bcc-208811f59b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.565299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1013/2057] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.577125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.578947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.647910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5c52c-aaa2-40e6-9f90-5b20c2bb365f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.653127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.653787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.654118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.654402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.656933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.726748 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ef0a2-b765-468d-8bcc-208811f59b37 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.728448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6f9ef0a2-b765-468d-8bcc-208811f59b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.783324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.786282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.802080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1045/2058] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:15.806478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.808466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:15.814243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.889871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ef0a2-b765-468d-8bcc-208811f59b37 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.898653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.899323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.899610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.901084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] PATH_INFO: `/v3/users/5b629c0a846d4cb3b390161f9581d133` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.905827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b629c0a846d4cb3b390161f9581d133)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.908702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.943195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e04513-4181-41be-8de2-ddad6cb214d7 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.949099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.951515 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.953047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.953471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] PATH_INFO: `/v3/projects/e67b9d2cb28f44c99e5d331f16c8a4d2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.955115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e67b9d2cb28f44c99e5d331f16c8a4d2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.959281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.977759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b629c0a846d4cb3b390161f9581d133'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:15.988786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b629c0a846d4cb3b390161f9581d133'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:15.989809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b629c0a846d4cb3b390161f9581d133'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.015778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b629c0a846d4cb3b390161f9581d133'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.027114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-833df96d-5b25-4c00-acfa-21450f290cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b629c0a846d4cb3b390161f9581d133'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.038783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e67b9d2cb28f44c99e5d331f16c8a4d2', 'request_id': 'req-32916007-c9f9-4548-af6f-800616361463'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.040286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1014/2059] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:15 2022] DELETE /identity/v3/users/5b629c0a846d4cb3b390161f9581d133 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:16.047423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1983-b94d-4782-b474-0f5e2e8ceb24 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.047956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.048168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1983-b94d-4782-b474-0f5e2e8ceb24 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.048457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad1983-b94d-4782-b474-0f5e2e8ceb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.050428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e67b9d2cb28f44c99e5d331f16c8a4d2', 'request_id': 'req-32916007-c9f9-4548-af6f-800616361463'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.059808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e67b9d2cb28f44c99e5d331f16c8a4d2', 'request_id': 'req-32916007-c9f9-4548-af6f-800616361463'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.077163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] The token cache is being invalidate because project e67b9d2cb28f44c99e5d331f16c8a4d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:16.077632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.078406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-32916007-c9f9-4548-af6f-800616361463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.080299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1046/2060] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:15 2022] DELETE /identity/v3/projects/e67b9d2cb28f44c99e5d331f16c8a4d2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:16.084979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-36ad1983-b94d-4782-b474-0f5e2e8ceb24 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:16.085364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd362e-e436-4e99-b927-88d92fe0f348 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.085738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd362e-e436-4e99-b927-88d92fe0f348 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.086089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd362e-e436-4e99-b927-88d92fe0f348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.086330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-36ad1983-b94d-4782-b474-0f5e2e8ceb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.116477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3cdd362e-e436-4e99-b927-88d92fe0f348 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:16.117853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3cdd362e-e436-4e99-b927-88d92fe0f348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.184978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1015/2061] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:16.204095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1983-b94d-4782-b474-0f5e2e8ceb24 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.205927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-36ad1983-b94d-4782-b474-0f5e2e8ceb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.233899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1047/2062] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:16.235752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9258-927e-46f9-8e99-06e0b71a1efa None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.236009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9258-927e-46f9-8e99-06e0b71a1efa None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.236247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae9258-927e-46f9-8e99-06e0b71a1efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.268418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-38ae9258-927e-46f9-8e99-06e0b71a1efa None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:16.270234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-38ae9258-927e-46f9-8e99-06e0b71a1efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.279505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1048/2063] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:16.283416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae9258-927e-46f9-8e99-06e0b71a1efa None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.295249 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae9258-927e-46f9-8e99-06e0b71a1efa None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.300535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.301148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.301432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.302239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.303291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:16.303602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.305585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.318932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad1983-b94d-4782-b474-0f5e2e8ceb24 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.319486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1049/2064] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:50:16.323865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f978ba-ac19-4a60-b587-f1654e3543d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.324038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.324920 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f978ba-ac19-4a60-b587-f1654e3543d2 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.325295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f978ba-ac19-4a60-b587-f1654e3543d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.325558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f978ba-ac19-4a60-b587-f1654e3543d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.337043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f978ba-ac19-4a60-b587-f1654e3543d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.338736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f978ba-ac19-4a60-b587-f1654e3543d2 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.341675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af2f56-4b9a-4c29-bb33-5c246e8a6fd6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.344639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.348632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8eb97-7575-4aac-8b64-29d278f89043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.349482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8eb97-7575-4aac-8b64-29d278f89043 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.349691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8eb97-7575-4aac-8b64-29d278f89043 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.349956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8eb97-7575-4aac-8b64-29d278f89043 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.350644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8eb97-7575-4aac-8b64-29d278f89043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.350842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1016/2065] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:16.353279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8eb97-7575-4aac-8b64-29d278f89043 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.392113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1050/2066] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/projects => generated 399 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:16.393795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f978ba-ac19-4a60-b587-f1654e3543d2 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.404746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f978ba-ac19-4a60-b587-f1654e3543d2 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.474797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26e24-9f76-47fa-86b5-0955575e728f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.475647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26e24-9f76-47fa-86b5-0955575e728f admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.475769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26e24-9f76-47fa-86b5-0955575e728f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.475966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b26e24-9f76-47fa-86b5-0955575e728f admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.476621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b26e24-9f76-47fa-86b5-0955575e728f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.478628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b26e24-9f76-47fa-86b5-0955575e728f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.505722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1017/2067] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:16.509146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8eb97-7575-4aac-8b64-29d278f89043 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.521111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8eb97-7575-4aac-8b64-29d278f89043 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.539475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37609-ba72-4a32-a8b3-2d0cfb2dff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.542027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37609-ba72-4a32-a8b3-2d0cfb2dff3a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.542524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37609-ba72-4a32-a8b3-2d0cfb2dff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.542694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37609-ba72-4a32-a8b3-2d0cfb2dff3a admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.543918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc37609-ba72-4a32-a8b3-2d0cfb2dff3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.546284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc37609-ba72-4a32-a8b3-2d0cfb2dff3a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.558550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1051/2068] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:16.562661 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26e24-9f76-47fa-86b5-0955575e728f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.573889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b26e24-9f76-47fa-86b5-0955575e728f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.579688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.580535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.580893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.581227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.582101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.583855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.594575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1018/2069] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:16.600119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37609-ba72-4a32-a8b3-2d0cfb2dff3a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.632981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37609-ba72-4a32-a8b3-2d0cfb2dff3a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.638832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0f10-a71f-46f0-9a89-048eb181c6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.639677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0f10-a71f-46f0-9a89-048eb181c6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.640142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0f10-a71f-46f0-9a89-048eb181c6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.640558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0f10-a71f-46f0-9a89-048eb181c6c9 admin admin] PATH_INFO: `/v3/projects/691c5334804c4da5bc616911ea6770dc/users/8f4462898a4d4a728ce1b751c942fd4a/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.663374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.665322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.665526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a0f10-a71f-46f0-9a89-048eb181c6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=691c5334804c4da5bc616911ea6770dc, user_id=8f4462898a4d4a728ce1b751c942fd4a, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.668764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a0f10-a71f-46f0-9a89-048eb181c6c9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.694399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d1ef5-1fc5-41c3-98a6-1d3307ba1d0e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.696051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1052/2070] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:16 2022] PUT /identity/v3/projects/691c5334804c4da5bc616911ea6770dc/users/8f4462898a4d4a728ce1b751c942fd4a/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:16.697905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bcdf4-8410-4641-8553-87f41a5db3da None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.698243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bcdf4-8410-4641-8553-87f41a5db3da None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.698448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bcdf4-8410-4641-8553-87f41a5db3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.727775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-489bcdf4-8410-4641-8553-87f41a5db3da None None] MFA Rules not processed for user `8f4462898a4d4a728ce1b751c942fd4a`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:16.728997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-489bcdf4-8410-4641-8553-87f41a5db3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.767080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.767829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.768110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.768715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] PATH_INFO: `/v3/projects/c5924a5fef0b4f75825f3ec03cb6999a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.770718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5924a5fef0b4f75825f3ec03cb6999a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.772782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.802504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.810540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1053/2071] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:16.823779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bcdf4-8410-4641-8553-87f41a5db3da None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.825327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-489bcdf4-8410-4641-8553-87f41a5db3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.832278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5924a5fef0b4f75825f3ec03cb6999a', 'request_id': 'req-504de5ea-f68e-426c-85e2-8aad685e28e4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.840800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5924a5fef0b4f75825f3ec03cb6999a', 'request_id': 'req-504de5ea-f68e-426c-85e2-8aad685e28e4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.852431 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5924a5fef0b4f75825f3ec03cb6999a', 'request_id': 'req-504de5ea-f68e-426c-85e2-8aad685e28e4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.871957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] The token cache is being invalidate because project c5924a5fef0b4f75825f3ec03cb6999a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:16.872381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.873429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:16.874968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1019/2072] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:16 2022] DELETE /identity/v3/projects/c5924a5fef0b4f75825f3ec03cb6999a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:16.879902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.881410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.893634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bcdf4-8410-4641-8553-87f41a5db3da None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.894444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.966278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.966903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.967180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.967453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.969563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.976585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504de5ea-f68e-426c-85e2-8aad685e28e4 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.983321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.984004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.984293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.985034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] PATH_INFO: `/v3/users/ae6431e791c34df98c131c78cc60c3b5` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.987079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6431e791c34df98c131c78cc60c3b5)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.995057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.060858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.062862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.073742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1054/2073] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:17.076894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6431e791c34df98c131c78cc60c3b5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.085834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6431e791c34df98c131c78cc60c3b5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.087016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6431e791c34df98c131c78cc60c3b5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.102933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6431e791c34df98c131c78cc60c3b5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.112939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6431e791c34df98c131c78cc60c3b5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.120787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1020/2074] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:16 2022] DELETE /identity/v3/users/ae6431e791c34df98c131c78cc60c3b5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:17.547004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:17.550621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.572119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cd514-df01-47c1-8598-6102f914cae2 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:17.652595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:17.654822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.673060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.674067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.674416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.674756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.677570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.725446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:17.784076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.785946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.787955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e6186-1c0d-498f-885a-0d7ffb8565f8 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:17.795004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.795871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.796347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.796729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] PATH_INFO: `/v3/projects/213aec7078ba468e95ef4f441af43730` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.798129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213aec7078ba468e95ef4f441af43730)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.799123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1055/2075] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:17.813162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.876875 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:17.903058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213aec7078ba468e95ef4f441af43730', 'request_id': 'req-522288f0-6b96-4115-b400-afcd9c9045f3'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.919635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213aec7078ba468e95ef4f441af43730', 'request_id': 'req-522288f0-6b96-4115-b400-afcd9c9045f3'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.927478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213aec7078ba468e95ef4f441af43730', 'request_id': 'req-522288f0-6b96-4115-b400-afcd9c9045f3'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.945964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] The token cache is being invalidate because project 213aec7078ba468e95ef4f441af43730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:17.946505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.947276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:17.953980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1021/2076] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:17 2022] DELETE /identity/v3/projects/213aec7078ba468e95ef4f441af43730 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:17.954555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:17.969844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db679307-4de1-463c-8804-80a0f48db549 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:18.067471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:18.069022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:18.069479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:18.070500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] PATH_INFO: `/v3/users/cccacd7a0f874a86b514f74654126493` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:18.071837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cccacd7a0f874a86b514f74654126493)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:18.073729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:18.145952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cccacd7a0f874a86b514f74654126493'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.163137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cccacd7a0f874a86b514f74654126493'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.163824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cccacd7a0f874a86b514f74654126493'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.185396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cccacd7a0f874a86b514f74654126493'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.193874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cccacd7a0f874a86b514f74654126493'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.202797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1056/2077] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:17 2022] DELETE /identity/v3/users/cccacd7a0f874a86b514f74654126493 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:18.528964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:18.530536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:18.565174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:18.567319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:18.574132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522288f0-6b96-4115-b400-afcd9c9045f3 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:18.657904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:18.699598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bd686-b255-42e5-952b-1f5a699ccd43 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:18.703707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5832648-65ba-401c-a155-3e8a73aef917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:18.704435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5832648-65ba-401c-a155-3e8a73aef917 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:18.704717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5832648-65ba-401c-a155-3e8a73aef917 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:18.704988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5832648-65ba-401c-a155-3e8a73aef917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:18.709015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f5832648-65ba-401c-a155-3e8a73aef917 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:18.716981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:18.717768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:18.718122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:18.718473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] PATH_INFO: `/v3/projects/88743c8e943c49f2891845179f9e250e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:18.719955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=88743c8e943c49f2891845179f9e250e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:18.721935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:18.794281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5832648-65ba-401c-a155-3e8a73aef917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:18.796906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5832648-65ba-401c-a155-3e8a73aef917 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:18.800016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88743c8e943c49f2891845179f9e250e', 'request_id': 'req-1a389ab4-8159-4400-8ee0-39b479043c4a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.810999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88743c8e943c49f2891845179f9e250e', 'request_id': 'req-1a389ab4-8159-4400-8ee0-39b479043c4a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.811553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1057/2078] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:18.838503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88743c8e943c49f2891845179f9e250e', 'request_id': 'req-1a389ab4-8159-4400-8ee0-39b479043c4a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.858463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] The token cache is being invalidate because project 88743c8e943c49f2891845179f9e250e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:18.859138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.859867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1a389ab4-8159-4400-8ee0-39b479043c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:18.863636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1022/2079] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:18 2022] DELETE /identity/v3/projects/88743c8e943c49f2891845179f9e250e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:18.867858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2043d-797e-4152-b36a-15fd6e8c5729 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:18.868471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2043d-797e-4152-b36a-15fd6e8c5729 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:18.868960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2043d-797e-4152-b36a-15fd6e8c5729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:18.905573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a8b2043d-797e-4152-b36a-15fd6e8c5729 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:18.908470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a8b2043d-797e-4152-b36a-15fd6e8c5729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:19.033908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1058/2080] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:19.042121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e0d3-56a3-4ec8-8b97-c8f87ca05699 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.042410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e0d3-56a3-4ec8-8b97-c8f87ca05699 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.042708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1482e0d3-56a3-4ec8-8b97-c8f87ca05699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.108093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1482e0d3-56a3-4ec8-8b97-c8f87ca05699 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:19.111471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1482e0d3-56a3-4ec8-8b97-c8f87ca05699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:19.131492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1023/2081] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:19.135340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2043d-797e-4152-b36a-15fd6e8c5729 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:19.148872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2043d-797e-4152-b36a-15fd6e8c5729 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:19.154844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:19.155522 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.155847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.156115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.156793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:19.157096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:19.158951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:19.188056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1059/2082] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:19.189705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482e0d3-56a3-4ec8-8b97-c8f87ca05699 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:19.201752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1482e0d3-56a3-4ec8-8b97-c8f87ca05699 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:19.207936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8378606-9659-4e1f-b28e-b703a00d6a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:19.208576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8378606-9659-4e1f-b28e-b703a00d6a8a admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.209479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8378606-9659-4e1f-b28e-b703a00d6a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.210467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8378606-9659-4e1f-b28e-b703a00d6a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.213303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8378606-9659-4e1f-b28e-b703a00d6a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:19.219550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8378606-9659-4e1f-b28e-b703a00d6a8a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:19.253400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1024/2083] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:19 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:19.258192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:19.269334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96773442-5c84-48d7-9660-686d32a1bc5b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:19.346421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603465dc-64b4-44ce-a2e5-0f1a6f3e4ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:19.347331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603465dc-64b4-44ce-a2e5-0f1a6f3e4ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.347711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603465dc-64b4-44ce-a2e5-0f1a6f3e4ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.348084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603465dc-64b4-44ce-a2e5-0f1a6f3e4ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.348936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603465dc-64b4-44ce-a2e5-0f1a6f3e4ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:19.351186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603465dc-64b4-44ce-a2e5-0f1a6f3e4ee7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:19.381663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1060/2084] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:19 2022] POST /identity/v3/users => generated 290 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:19.385133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8378606-9659-4e1f-b28e-b703a00d6a8a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:19.401217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8378606-9659-4e1f-b28e-b703a00d6a8a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:19.410830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb400fb-159b-4fea-8983-624ef1246fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:19.411449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb400fb-159b-4fea-8983-624ef1246fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.411725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb400fb-159b-4fea-8983-624ef1246fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.412616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb400fb-159b-4fea-8983-624ef1246fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.413278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb400fb-159b-4fea-8983-624ef1246fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:19.414931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb400fb-159b-4fea-8983-624ef1246fc4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:19.429778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1025/2085] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:19.431448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603465dc-64b4-44ce-a2e5-0f1a6f3e4ee7 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:19.444744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603465dc-64b4-44ce-a2e5-0f1a6f3e4ee7 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:19.450241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141101a-71c5-4b62-ae68-72ca962bcc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:19.451384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141101a-71c5-4b62-ae68-72ca962bcc9e admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.451840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141101a-71c5-4b62-ae68-72ca962bcc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.452170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141101a-71c5-4b62-ae68-72ca962bcc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.452906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141101a-71c5-4b62-ae68-72ca962bcc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:19.454819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1141101a-71c5-4b62-ae68-72ca962bcc9e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:19.458030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:19.473817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1061/2086] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:19.476049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb400fb-159b-4fea-8983-624ef1246fc4 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:19.490580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb400fb-159b-4fea-8983-624ef1246fc4 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:19.500685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:19.501367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.501678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.501963 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] PATH_INFO: `/v3/projects/1f154705e31748d9b7b8455ee5f73b70/users/1e09b3554bb64828b857632d1de602d8/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.506334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:19.522460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f154705e31748d9b7b8455ee5f73b70, user_id=1e09b3554bb64828b857632d1de602d8, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:19.525464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:19.542708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1026/2087] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:19 2022] PUT /identity/v3/projects/1f154705e31748d9b7b8455ee5f73b70/users/1e09b3554bb64828b857632d1de602d8/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:19.548537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746247e0-651d-4303-80a7-3ae43b79bc6b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.549287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746247e0-651d-4303-80a7-3ae43b79bc6b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.549754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746247e0-651d-4303-80a7-3ae43b79bc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.573512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-746247e0-651d-4303-80a7-3ae43b79bc6b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:19.574772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-746247e0-651d-4303-80a7-3ae43b79bc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:19.657372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1062/2088] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:19.663984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:19.665723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:19.725313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14c7c1-06eb-479b-9dae-292e8020a2e0 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:19.729294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ace5e-8460-4452-98cf-f55b0e3df061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:19.729948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ace5e-8460-4452-98cf-f55b0e3df061 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.730231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ace5e-8460-4452-98cf-f55b0e3df061 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.730508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ace5e-8460-4452-98cf-f55b0e3df061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.739878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ace5e-8460-4452-98cf-f55b0e3df061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:19.741331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ace5e-8460-4452-98cf-f55b0e3df061 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:19.750883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1027/2089] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:19.872179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82390-eab9-437b-80bc-45ba13124936 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.872516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82390-eab9-437b-80bc-45ba13124936 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.872812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e82390-eab9-437b-80bc-45ba13124936 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.874353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1063/2090] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:19.884768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe157c4-d7a2-497a-aa3e-77161fbafa14 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:19.885386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe157c4-d7a2-497a-aa3e-77161fbafa14 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:19.885905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe157c4-d7a2-497a-aa3e-77161fbafa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:19.919787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-cbe157c4-d7a2-497a-aa3e-77161fbafa14 None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:19.921368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cbe157c4-d7a2-497a-aa3e-77161fbafa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:19.994895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82390-eab9-437b-80bc-45ba13124936 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:19.999205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c8e82390-eab9-437b-80bc-45ba13124936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:20.023326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e82390-eab9-437b-80bc-45ba13124936 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:20.030051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:20.032413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.033246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.033755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] PATH_INFO: `/v3/users/0cf74b3dd4c24c1aa3c0ac659b37c0e1` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.035310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf74b3dd4c24c1aa3c0ac659b37c0e1)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:20.037507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:20.051032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1028/2091] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:20.071602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe157c4-d7a2-497a-aa3e-77161fbafa14 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:20.073438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cbe157c4-d7a2-497a-aa3e-77161fbafa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:20.099750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe157c4-d7a2-497a-aa3e-77161fbafa14 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:20.108825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c595d-1160-4a36-b8bf-71d6595d133a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:20.109758 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c595d-1160-4a36-b8bf-71d6595d133a service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.110234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c595d-1160-4a36-b8bf-71d6595d133a service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.111191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c595d-1160-4a36-b8bf-71d6595d133a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.123344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c595d-1160-4a36-b8bf-71d6595d133a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:20.126323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c595d-1160-4a36-b8bf-71d6595d133a service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:20.142670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf74b3dd4c24c1aa3c0ac659b37c0e1'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.153695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1029/2092] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:20.167810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf74b3dd4c24c1aa3c0ac659b37c0e1'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.168459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf74b3dd4c24c1aa3c0ac659b37c0e1'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.184600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf74b3dd4c24c1aa3c0ac659b37c0e1'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.199390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-645421a8-aaaf-49b2-a4fc-c4bff7a8480f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf74b3dd4c24c1aa3c0ac659b37c0e1'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.216056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1064/2093] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:19 2022] DELETE /identity/v3/users/0cf74b3dd4c24c1aa3c0ac659b37c0e1 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:20.511581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c595d-1160-4a36-b8bf-71d6595d133a service placement] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:20.526985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c595d-1160-4a36-b8bf-71d6595d133a service placement] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:20.596450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:20.597637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.597933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.598213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] PATH_INFO: `/v3/projects/f2c073eb9c7a494ab879f3af8cc96717` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.599983 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c073eb9c7a494ab879f3af8cc96717)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:20.602022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:20.614381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:20.653537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c073eb9c7a494ab879f3af8cc96717', 'request_id': 'req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.660717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c073eb9c7a494ab879f3af8cc96717', 'request_id': 'req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.668238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c073eb9c7a494ab879f3af8cc96717', 'request_id': 'req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.686127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] The token cache is being invalidate because project f2c073eb9c7a494ab879f3af8cc96717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:20.686373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.686977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4eeaae51-3b1d-43c0-a3cb-c1ef2ea68dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.689967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1030/2094] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:20 2022] DELETE /identity/v3/projects/f2c073eb9c7a494ab879f3af8cc96717 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:20.695213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e483-cb78-445d-b2af-5f92f622bbae None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.695583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e483-cb78-445d-b2af-5f92f622bbae None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.695866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea3e483-cb78-445d-b2af-5f92f622bbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.719105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-1ea3e483-cb78-445d-b2af-5f92f622bbae None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:20.720499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1ea3e483-cb78-445d-b2af-5f92f622bbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:20.801023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1065/2095] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:20.803251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78591d-0bf0-4455-85b9-fe13762edf87 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.803813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78591d-0bf0-4455-85b9-fe13762edf87 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.803813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78591d-0bf0-4455-85b9-fe13762edf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.825637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ef78591d-0bf0-4455-85b9-fe13762edf87 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:20.826857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ef78591d-0bf0-4455-85b9-fe13762edf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:20.838368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1031/2096] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:20.842823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3e483-cb78-445d-b2af-5f92f622bbae None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:20.853532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea3e483-cb78-445d-b2af-5f92f622bbae None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:20.858396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:20.858985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.859426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.859677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.860341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:20.860650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:20.863974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:20.872975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:20.883350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1066/2097] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:20.884785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78591d-0bf0-4455-85b9-fe13762edf87 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:20.897669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78591d-0bf0-4455-85b9-fe13762edf87 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:20.903080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d1ad-3591-474c-92ee-0eeb21ea5638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:20.903698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d1ad-3591-474c-92ee-0eeb21ea5638 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.903986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d1ad-3591-474c-92ee-0eeb21ea5638 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.904251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76d1ad-3591-474c-92ee-0eeb21ea5638 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.904891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d1ad-3591-474c-92ee-0eeb21ea5638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:20.906858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f76d1ad-3591-474c-92ee-0eeb21ea5638 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:20.936225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1032/2098] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:20 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:20.940374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:20.951062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdec35-56b1-433a-b284-f32dd9b69b08 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:21.014727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bbb65-48fc-4dc9-a421-69d07c7f2c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:21.016183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bbb65-48fc-4dc9-a421-69d07c7f2c04 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:21.016672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bbb65-48fc-4dc9-a421-69d07c7f2c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:21.017144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3bbb65-48fc-4dc9-a421-69d07c7f2c04 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:21.018476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bbb65-48fc-4dc9-a421-69d07c7f2c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:21.022191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3bbb65-48fc-4dc9-a421-69d07c7f2c04 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:21.048964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1067/2099] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:20 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:21.055380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d1ad-3591-474c-92ee-0eeb21ea5638 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:21.069391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76d1ad-3591-474c-92ee-0eeb21ea5638 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:21.075567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913afdf-a586-404c-837c-cd9a56ea8759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:21.076402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913afdf-a586-404c-837c-cd9a56ea8759 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:21.076811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913afdf-a586-404c-837c-cd9a56ea8759 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:21.077226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913afdf-a586-404c-837c-cd9a56ea8759 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:21.078176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913afdf-a586-404c-837c-cd9a56ea8759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:21.080522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913afdf-a586-404c-837c-cd9a56ea8759 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:21.092944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1033/2100] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:21.096942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bbb65-48fc-4dc9-a421-69d07c7f2c04 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:21.107963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3bbb65-48fc-4dc9-a421-69d07c7f2c04 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:21.113947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc51e46-f90f-45ac-9cb1-30b77bee537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:21.114555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc51e46-f90f-45ac-9cb1-30b77bee537a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:21.114840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc51e46-f90f-45ac-9cb1-30b77bee537a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:21.115443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc51e46-f90f-45ac-9cb1-30b77bee537a admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:21.115762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc51e46-f90f-45ac-9cb1-30b77bee537a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:21.117360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc51e46-f90f-45ac-9cb1-30b77bee537a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:21.135595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1068/2101] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:21.137372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913afdf-a586-404c-837c-cd9a56ea8759 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:21.148967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913afdf-a586-404c-837c-cd9a56ea8759 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:21.153975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:21.154695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:21.155638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:21.155919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] PATH_INFO: `/v3/projects/f56d4e10297e42689603cf6cae0eeb40/users/ff83f512018b4053bdf4653d1782d7bc/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:21.174950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f56d4e10297e42689603cf6cae0eeb40, user_id=ff83f512018b4053bdf4653d1782d7bc, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:21.177637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:21.208898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1034/2102] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:21 2022] PUT /identity/v3/projects/f56d4e10297e42689603cf6cae0eeb40/users/ff83f512018b4053bdf4653d1782d7bc/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:21.211601 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b61e2-d73a-4289-933d-351799e11ef0 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:21.212505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b61e2-d73a-4289-933d-351799e11ef0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:21.212965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b61e2-d73a-4289-933d-351799e11ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:21.262164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9c2b61e2-d73a-4289-933d-351799e11ef0 None None] MFA Rules not processed for user `ff83f512018b4053bdf4653d1782d7bc`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:21.263452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9c2b61e2-d73a-4289-933d-351799e11ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:21.378246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1069/2103] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:21.392849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:21.395172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:21.517913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d8eb-ad0e-4a7d-9480-d17fd157ccc8 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:21.541800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f75ece-98fe-47ec-9cc2-ed78e2bda02f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:21.541800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f75ece-98fe-47ec-9cc2-ed78e2bda02f service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:21.541800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f75ece-98fe-47ec-9cc2-ed78e2bda02f service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:21.541800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f75ece-98fe-47ec-9cc2-ed78e2bda02f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:21.549803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f75ece-98fe-47ec-9cc2-ed78e2bda02f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:21.550854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f75ece-98fe-47ec-9cc2-ed78e2bda02f service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:21.563354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1035/2104] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:21.870098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b61e2-d73a-4289-933d-351799e11ef0 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:21.962440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b61e2-d73a-4289-933d-351799e11ef0 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:21.968445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:21.969175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:21.969659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:21.970191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] PATH_INFO: `/v3/users/ff83f512018b4053bdf4653d1782d7bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:21.971862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff83f512018b4053bdf4653d1782d7bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:21.974253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:21.981963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:22.065339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff83f512018b4053bdf4653d1782d7bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.075823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff83f512018b4053bdf4653d1782d7bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.076609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff83f512018b4053bdf4653d1782d7bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.092897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff83f512018b4053bdf4653d1782d7bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.101839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff83f512018b4053bdf4653d1782d7bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.107990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1070/2105] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:21 2022] DELETE /identity/v3/users/ff83f512018b4053bdf4653d1782d7bc => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:22.112399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9d194-7aff-4dc3-a939-2d727dfe646a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:22.112942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9d194-7aff-4dc3-a939-2d727dfe646a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:22.113102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9d194-7aff-4dc3-a939-2d727dfe646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:22.142608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-a3b9d194-7aff-4dc3-a939-2d727dfe646a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:22.143831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a3b9d194-7aff-4dc3-a939-2d727dfe646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:22.161261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:22.235465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1036/2106] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:22.257985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:22.263199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:22.360910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529515f9-26a6-42fb-94ea-fc3a3c75a4d4 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:22.366649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341f170-8657-4fe6-9d1a-cda0a6b11c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:22.367578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341f170-8657-4fe6-9d1a-cda0a6b11c15 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:22.368037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341f170-8657-4fe6-9d1a-cda0a6b11c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:22.368400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341f170-8657-4fe6-9d1a-cda0a6b11c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:22.387088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341f170-8657-4fe6-9d1a-cda0a6b11c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:22.389810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341f170-8657-4fe6-9d1a-cda0a6b11c15 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:22.403505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1071/2107] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:22.999681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9d194-7aff-4dc3-a939-2d727dfe646a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.001153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a3b9d194-7aff-4dc3-a939-2d727dfe646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:23.021221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9d194-7aff-4dc3-a939-2d727dfe646a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:23.031535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:23.032168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.032451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.032726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] PATH_INFO: `/v3/projects/f56d4e10297e42689603cf6cae0eeb40` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.034061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f56d4e10297e42689603cf6cae0eeb40)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:23.036565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.107309 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f56d4e10297e42689603cf6cae0eeb40', 'request_id': 'req-61749696-63e7-487a-9d17-7457e184db53'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:23.115240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f56d4e10297e42689603cf6cae0eeb40', 'request_id': 'req-61749696-63e7-487a-9d17-7457e184db53'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:23.122923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f56d4e10297e42689603cf6cae0eeb40', 'request_id': 'req-61749696-63e7-487a-9d17-7457e184db53'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:23.150527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] The token cache is being invalidate because project f56d4e10297e42689603cf6cae0eeb40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:23.150958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:23.151749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-61749696-63e7-487a-9d17-7457e184db53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:23.158016 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1037/2108] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:22 2022] DELETE /identity/v3/projects/f56d4e10297e42689603cf6cae0eeb40 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:23.159838 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537b691-7bdc-415b-9b11-12b6d70ccb09 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.160118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537b691-7bdc-415b-9b11-12b6d70ccb09 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.160442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537b691-7bdc-415b-9b11-12b6d70ccb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.182756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4537b691-7bdc-415b-9b11-12b6d70ccb09 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:23.184906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4537b691-7bdc-415b-9b11-12b6d70ccb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:23.299086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1072/2109] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:23.309133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f50a5-7b85-4184-88b0-8f0b1f9edc6e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.309736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f50a5-7b85-4184-88b0-8f0b1f9edc6e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.310356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f50a5-7b85-4184-88b0-8f0b1f9edc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.367162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e36f50a5-7b85-4184-88b0-8f0b1f9edc6e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:23.375598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e36f50a5-7b85-4184-88b0-8f0b1f9edc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:23.400419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1038/2110] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:23.404663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537b691-7bdc-415b-9b11-12b6d70ccb09 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.414886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537b691-7bdc-415b-9b11-12b6d70ccb09 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:23.425283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:23.426208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.426633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.427046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.427864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:23.428310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:23.430097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.458009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1073/2111] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:23.461538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f50a5-7b85-4184-88b0-8f0b1f9edc6e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.474327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f50a5-7b85-4184-88b0-8f0b1f9edc6e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:23.487401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde91a-1da5-4e13-8b55-ab329c543c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:23.488257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde91a-1da5-4e13-8b55-ab329c543c9c admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.488802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde91a-1da5-4e13-8b55-ab329c543c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.489218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cde91a-1da5-4e13-8b55-ab329c543c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.490092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cde91a-1da5-4e13-8b55-ab329c543c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:23.493797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cde91a-1da5-4e13-8b55-ab329c543c9c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.550876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1039/2112] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:23 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:23.552030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.565584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f48fe9-3f37-4d24-a7b5-93f359f06c66 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:23.587527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:23.645833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48905ac9-cee9-4911-8821-cbd7e84fc95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:23.646664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48905ac9-cee9-4911-8821-cbd7e84fc95b admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.647011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48905ac9-cee9-4911-8821-cbd7e84fc95b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.647463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48905ac9-cee9-4911-8821-cbd7e84fc95b admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.648315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48905ac9-cee9-4911-8821-cbd7e84fc95b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:23.651119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48905ac9-cee9-4911-8821-cbd7e84fc95b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.685944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1074/2113] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:23 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:23.690075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde91a-1da5-4e13-8b55-ab329c543c9c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.713410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cde91a-1da5-4e13-8b55-ab329c543c9c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:23.721726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36336eec-d3ab-438a-8218-bdc82affd94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:23.722587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336eec-d3ab-438a-8218-bdc82affd94e admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.722968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336eec-d3ab-438a-8218-bdc82affd94e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.723505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36336eec-d3ab-438a-8218-bdc82affd94e admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.724548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36336eec-d3ab-438a-8218-bdc82affd94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:23.726522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36336eec-d3ab-438a-8218-bdc82affd94e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.738953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1040/2114] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:23.744950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48905ac9-cee9-4911-8821-cbd7e84fc95b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.755534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48905ac9-cee9-4911-8821-cbd7e84fc95b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:23.761353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686b001-711f-4a0e-9271-5c116926238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:23.762091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b001-711f-4a0e-9271-5c116926238f admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.762464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b001-711f-4a0e-9271-5c116926238f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.762837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b001-711f-4a0e-9271-5c116926238f admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.763600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686b001-711f-4a0e-9271-5c116926238f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:23.765734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686b001-711f-4a0e-9271-5c116926238f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.781291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1075/2115] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:23.783038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36336eec-d3ab-438a-8218-bdc82affd94e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.796256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36336eec-d3ab-438a-8218-bdc82affd94e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:23.803680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813c0bb-a545-4c61-ba98-342d22541074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:23.804925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813c0bb-a545-4c61-ba98-342d22541074 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.805307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813c0bb-a545-4c61-ba98-342d22541074 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.805712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813c0bb-a545-4c61-ba98-342d22541074 admin admin] PATH_INFO: `/v3/projects/6d252fa109fd4829b1d4337b0aa7fcf5/users/568a96d42f0f47a98b4eddf02c617271/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.825248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813c0bb-a545-4c61-ba98-342d22541074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d252fa109fd4829b1d4337b0aa7fcf5, user_id=568a96d42f0f47a98b4eddf02c617271, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:23.830035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813c0bb-a545-4c61-ba98-342d22541074 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.850416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1041/2116] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:23 2022] PUT /identity/v3/projects/6d252fa109fd4829b1d4337b0aa7fcf5/users/568a96d42f0f47a98b4eddf02c617271/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:23.861295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112bc5-cfc0-48d7-84f7-4dfec6797451 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.861981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112bc5-cfc0-48d7-84f7-4dfec6797451 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.862342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112bc5-cfc0-48d7-84f7-4dfec6797451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.914120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4a112bc5-cfc0-48d7-84f7-4dfec6797451 None None] MFA Rules not processed for user `568a96d42f0f47a98b4eddf02c617271`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:23.916114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4a112bc5-cfc0-48d7-84f7-4dfec6797451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.056250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1076/2117] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:24.060513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813c0bb-a545-4c61-ba98-342d22541074 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.120290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:24.150492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813c0bb-a545-4c61-ba98-342d22541074 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.156678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.157567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.158164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.158541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.160960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.163119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.218703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1042/2118] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:24 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:24.221690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112bc5-cfc0-48d7-84f7-4dfec6797451 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.254146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112bc5-cfc0-48d7-84f7-4dfec6797451 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.262910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.264752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.373848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8245508-c31b-422b-b208-88fccd9547be admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.379811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7ed6f-d496-413e-8911-514c5ac5156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.380674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7ed6f-d496-413e-8911-514c5ac5156f admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.381149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7ed6f-d496-413e-8911-514c5ac5156f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.381664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7ed6f-d496-413e-8911-514c5ac5156f admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.381890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.382500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7ed6f-d496-413e-8911-514c5ac5156f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.382926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.383554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.384032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] PATH_INFO: `/v3/users/3f887898ce93492d80a2ed658d9b8641` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.385415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c7ed6f-d496-413e-8911-514c5ac5156f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.386943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f887898ce93492d80a2ed658d9b8641)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.389918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.432226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1077/2119] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:24 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:24.436133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7ed6f-d496-413e-8911-514c5ac5156f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.450438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c7ed6f-d496-413e-8911-514c5ac5156f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.457321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.458061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.458407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.458775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.459806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.461710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.463957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f887898ce93492d80a2ed658d9b8641'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.482656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f887898ce93492d80a2ed658d9b8641'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.483879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f887898ce93492d80a2ed658d9b8641'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.489457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1078/2120] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:24.493583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.496877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.508897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f887898ce93492d80a2ed658d9b8641'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.510957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:24.522487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aeed7-4f4b-4945-8cec-1816e8e27261 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.525538 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fdba3849-9d67-4951-b3e3-095b4c661db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f887898ce93492d80a2ed658d9b8641'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.534669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37686812-77ed-448c-8c62-2ca9ce46b2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.536098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37686812-77ed-448c-8c62-2ca9ce46b2f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.536382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37686812-77ed-448c-8c62-2ca9ce46b2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.536658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37686812-77ed-448c-8c62-2ca9ce46b2f0 admin admin] PATH_INFO: `/v3/projects/1ffaa7ac969848c993a521e63e0657b1/users/c216d1c6d8e5440d9ad80b598f57bdff/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.539143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1043/2121] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:24 2022] DELETE /identity/v3/users/3f887898ce93492d80a2ed658d9b8641 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:24.544094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce40bb0-4fe4-4b27-8838-6bae8e7bae0e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.545242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce40bb0-4fe4-4b27-8838-6bae8e7bae0e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.545242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce40bb0-4fe4-4b27-8838-6bae8e7bae0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.570206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37686812-77ed-448c-8c62-2ca9ce46b2f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ffaa7ac969848c993a521e63e0657b1, user_id=c216d1c6d8e5440d9ad80b598f57bdff, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.571877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3ce40bb0-4fe4-4b27-8838-6bae8e7bae0e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:24.573797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3ce40bb0-4fe4-4b27-8838-6bae8e7bae0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.574350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37686812-77ed-448c-8c62-2ca9ce46b2f0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.599211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1079/2122] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:24 2022] PUT /identity/v3/projects/1ffaa7ac969848c993a521e63e0657b1/users/c216d1c6d8e5440d9ad80b598f57bdff/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:24.603367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37686812-77ed-448c-8c62-2ca9ce46b2f0 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.660686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:24.702195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1044/2123] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:24.707626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37686812-77ed-448c-8c62-2ca9ce46b2f0 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.712550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce40bb0-4fe4-4b27-8838-6bae8e7bae0e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.715856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3ce40bb0-4fe4-4b27-8838-6bae8e7bae0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.719920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573b684-c42a-4cfb-abc0-7bb8d290f64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.720885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573b684-c42a-4cfb-abc0-7bb8d290f64e admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.722796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573b684-c42a-4cfb-abc0-7bb8d290f64e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.723345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573b684-c42a-4cfb-abc0-7bb8d290f64e admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.726788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573b684-c42a-4cfb-abc0-7bb8d290f64e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.729372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573b684-c42a-4cfb-abc0-7bb8d290f64e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.744693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1080/2124] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:24.750742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573b684-c42a-4cfb-abc0-7bb8d290f64e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.785225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573b684-c42a-4cfb-abc0-7bb8d290f64e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.796917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.798148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.807307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.808030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] PATH_INFO: `/v3/projects/1ffaa7ac969848c993a521e63e0657b1/users/c216d1c6d8e5440d9ad80b598f57bdff/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.810974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ffaa7ac969848c993a521e63e0657b1, user_id=c216d1c6d8e5440d9ad80b598f57bdff, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.825237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce40bb0-4fe4-4b27-8838-6bae8e7bae0e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.827372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.846250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1081/2125] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:24 2022] PUT /identity/v3/projects/1ffaa7ac969848c993a521e63e0657b1/users/c216d1c6d8e5440d9ad80b598f57bdff/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:24.848124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.862591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.902771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70f013-bbe9-4299-a611-e3b0876e5668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.903804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70f013-bbe9-4299-a611-e3b0876e5668 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.904849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70f013-bbe9-4299-a611-e3b0876e5668 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.906074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70f013-bbe9-4299-a611-e3b0876e5668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.921381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70f013-bbe9-4299-a611-e3b0876e5668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.923383 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70f013-bbe9-4299-a611-e3b0876e5668 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.949987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1045/2126] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:24.953068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcedfa-e616-4173-a212-c52d2b863027 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.953357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcedfa-e616-4173-a212-c52d2b863027 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.954631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcedfa-e616-4173-a212-c52d2b863027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.001012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-25bcedfa-e616-4173-a212-c52d2b863027 None None] MFA Rules not processed for user `c216d1c6d8e5440d9ad80b598f57bdff`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:25.002307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9391f-8ab2-4548-8eb2-5d4844d87bb5 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.002864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-25bcedfa-e616-4173-a212-c52d2b863027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.007975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.008796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.009080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.009359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] PATH_INFO: `/v3/users/8f4462898a4d4a728ce1b751c942fd4a` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.010916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4462898a4d4a728ce1b751c942fd4a)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.016030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.096074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4462898a4d4a728ce1b751c942fd4a'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.112224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4462898a4d4a728ce1b751c942fd4a'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.113399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4462898a4d4a728ce1b751c942fd4a'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.127275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:25.139605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4462898a4d4a728ce1b751c942fd4a'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.147572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2abccd16-7a23-4f75-9fa5-177d2bcc2755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4462898a4d4a728ce1b751c942fd4a'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.157222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1082/2127] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:24 2022] DELETE /identity/v3/users/8f4462898a4d4a728ce1b751c942fd4a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:25.157222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdee8f-7d7d-4e8e-90ca-4fcb8fa41f20 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.157222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdee8f-7d7d-4e8e-90ca-4fcb8fa41f20 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.157222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdee8f-7d7d-4e8e-90ca-4fcb8fa41f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.169401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1046/2128] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:25.172355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731dbb1-0709-4c0b-8971-a1a912e5f184 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.172725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731dbb1-0709-4c0b-8971-a1a912e5f184 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.184385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731dbb1-0709-4c0b-8971-a1a912e5f184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.205742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e731dbb1-0709-4c0b-8971-a1a912e5f184 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:25.207546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e731dbb1-0709-4c0b-8971-a1a912e5f184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.216193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fffdee8f-7d7d-4e8e-90ca-4fcb8fa41f20 None None] MFA Rules not processed for user `1e09b3554bb64828b857632d1de602d8`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:25.217748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fffdee8f-7d7d-4e8e-90ca-4fcb8fa41f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.241159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:25.366747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1083/2129] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:25.368479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdee8f-7d7d-4e8e-90ca-4fcb8fa41f20 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.369979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fffdee8f-7d7d-4e8e-90ca-4fcb8fa41f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.403402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1047/2130] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:25.418310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731dbb1-0709-4c0b-8971-a1a912e5f184 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.425101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e731dbb1-0709-4c0b-8971-a1a912e5f184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.447382 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731dbb1-0709-4c0b-8971-a1a912e5f184 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.454845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.455394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.455696 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.456655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.457300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.460316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.484669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdee8f-7d7d-4e8e-90ca-4fcb8fa41f20 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.489170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.489823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.490101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.490369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.505720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1048/2131] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:25.506693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.508392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.579519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.581014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.586583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e2c4f-bcf4-4703-a6e2-6dc8c8e051bc admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.591351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.591990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.592262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1084/2132] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:25.592389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.592621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] PATH_INFO: `/v3/projects/dd7c0f339f2b436a80010b06132e6822` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.593103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.593799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7c0f339f2b436a80010b06132e6822)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.594490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.595969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.621421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:25.655491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7c0f339f2b436a80010b06132e6822', 'request_id': 'req-ac16817f-70ca-47a8-9c8b-956a2c51b194'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.663499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7c0f339f2b436a80010b06132e6822', 'request_id': 'req-ac16817f-70ca-47a8-9c8b-956a2c51b194'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.679001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7c0f339f2b436a80010b06132e6822', 'request_id': 'req-ac16817f-70ca-47a8-9c8b-956a2c51b194'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.681283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded7bef-3777-4229-ab93-71040c9eefd5 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.698000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] The token cache is being invalidate because project dd7c0f339f2b436a80010b06132e6822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:25.698421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.699248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.700856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1049/2133] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:25 2022] DELETE /identity/v3/projects/dd7c0f339f2b436a80010b06132e6822 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:25.701709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.703197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.747133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.747745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.748022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.748299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.750132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.758462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:25.788326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16817f-70ca-47a8-9c8b-956a2c51b194 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.793481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.794133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.794415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.794688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.795362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.797492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.820701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1050/2134] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:25.821643 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.826765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.832967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.835113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.842084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcbc96-3e88-4868-90dc-87e4fa1ec96b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.846639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1085/2135] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:25.847537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.848223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.848929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.849174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.849540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.849908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] PATH_INFO: `/v3/users/568a96d42f0f47a98b4eddf02c617271` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.851195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=568a96d42f0f47a98b4eddf02c617271)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.853273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.873324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa48a2f-ec77-4c63-afc9-efc6863570b8 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.885575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.886445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.886927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.889551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] PATH_INFO: `/v3/users/1b8ea963087b4536862f39e0f1c8ad49` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.914321 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8ea963087b4536862f39e0f1c8ad49)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.917156 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.959299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '568a96d42f0f47a98b4eddf02c617271'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.970227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '568a96d42f0f47a98b4eddf02c617271'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.971289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '568a96d42f0f47a98b4eddf02c617271'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.988984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '568a96d42f0f47a98b4eddf02c617271'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.990554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8ea963087b4536862f39e0f1c8ad49'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:25.999226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '568a96d42f0f47a98b4eddf02c617271'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.003059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8ea963087b4536862f39e0f1c8ad49'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.004606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8ea963087b4536862f39e0f1c8ad49'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.022131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1051/2136] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:25 2022] DELETE /identity/v3/users/568a96d42f0f47a98b4eddf02c617271 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:26.022131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.022753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.027887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8ea963087b4536862f39e0f1c8ad49'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.036552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-dd3f26cd-9881-4f34-b847-e8ab2de8944e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8ea963087b4536862f39e0f1c8ad49'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.039347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:26.054997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1086/2137] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:25 2022] DELETE /identity/v3/users/1b8ea963087b4536862f39e0f1c8ad49 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:26.056520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d057-6e4a-43d9-913b-41e0f3cf33f2 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.056931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d057-6e4a-43d9-913b-41e0f3cf33f2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.057312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d057-6e4a-43d9-913b-41e0f3cf33f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.111849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a7c9d057-6e4a-43d9-913b-41e0f3cf33f2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:26.114428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a7c9d057-6e4a-43d9-913b-41e0f3cf33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.127549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4632410-ee43-4d65-805e-b56f91f8fab0 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.226842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:26.250468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1087/2138] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:26.257915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.258736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.259095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.259496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.260277 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.262360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.270194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9d057-6e4a-43d9-913b-41e0f3cf33f2 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.271703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a7c9d057-6e4a-43d9-913b-41e0f3cf33f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.297145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1052/2139] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:26.298828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.302250 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.342658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4576dce3-c731-4435-a92b-1e779f99dd8f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.349202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.349970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.350370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.350651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] PATH_INFO: `/v3/projects/691c5334804c4da5bc616911ea6770dc` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.351951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=691c5334804c4da5bc616911ea6770dc)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.354031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.410785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9d057-6e4a-43d9-913b-41e0f3cf33f2 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.420610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '691c5334804c4da5bc616911ea6770dc', 'request_id': 'req-512fe418-b803-472e-950e-4161e3100f12'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.432002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '691c5334804c4da5bc616911ea6770dc', 'request_id': 'req-512fe418-b803-472e-950e-4161e3100f12'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.448834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '691c5334804c4da5bc616911ea6770dc', 'request_id': 'req-512fe418-b803-472e-950e-4161e3100f12'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.469241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] The token cache is being invalidate because project 691c5334804c4da5bc616911ea6770dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:26.469600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.470360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:26.484240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.485438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.486402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.486896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.489571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1053/2140] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:26 2022] DELETE /identity/v3/projects/691c5334804c4da5bc616911ea6770dc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:26.490692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.491653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.506897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.710316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fe418-b803-472e-950e-4161e3100f12 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.716513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3958e34-99c2-44de-9290-607905f37419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.717314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3958e34-99c2-44de-9290-607905f37419 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.717826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3958e34-99c2-44de-9290-607905f37419 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.718176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3958e34-99c2-44de-9290-607905f37419 admin admin] PATH_INFO: `/v3/projects/5596b9569d0d499dabb635403998c294/users/9cc388e0a26444f8a17601d762d58d07/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.726304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.730206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.741652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3958e34-99c2-44de-9290-607905f37419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5596b9569d0d499dabb635403998c294, user_id=9cc388e0a26444f8a17601d762d58d07, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.747417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3958e34-99c2-44de-9290-607905f37419 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.752187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1088/2141] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:26.752944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.754532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.765304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:26.772021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2b2e6-755e-409c-ac30-c263fe7bd386 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.776321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1054/2142] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:26 2022] PUT /identity/v3/projects/5596b9569d0d499dabb635403998c294/users/9cc388e0a26444f8a17601d762d58d07/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:26.781551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3958e34-99c2-44de-9290-607905f37419 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.892684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:26.907928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3958e34-99c2-44de-9290-607905f37419 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.916408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0479d-759e-49be-9d03-a171d20099a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.917149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0479d-759e-49be-9d03-a171d20099a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.917503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0479d-759e-49be-9d03-a171d20099a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.919373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0479d-759e-49be-9d03-a171d20099a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.919719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.920330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.920610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.920761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0479d-759e-49be-9d03-a171d20099a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.920885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] PATH_INFO: `/v3/projects/d58a87728da34b67a63454017e7dc3fd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.922068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58a87728da34b67a63454017e7dc3fd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.923222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0479d-759e-49be-9d03-a171d20099a2 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.924272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.937238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1055/2143] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:26.942479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0479d-759e-49be-9d03-a171d20099a2 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.956623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0479d-759e-49be-9d03-a171d20099a2 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.963878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.971632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.971632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.971632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] PATH_INFO: `/v3/projects/5596b9569d0d499dabb635403998c294/users/9cc388e0a26444f8a17601d762d58d07/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.973796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5596b9569d0d499dabb635403998c294, user_id=9cc388e0a26444f8a17601d762d58d07, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.978088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.994062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1056/2144] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:26 2022] PUT /identity/v3/projects/5596b9569d0d499dabb635403998c294/users/9cc388e0a26444f8a17601d762d58d07/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:27.009708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58a87728da34b67a63454017e7dc3fd', 'request_id': 'req-3dd7fb71-4083-454e-9a41-fe63bcf79a48'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.027463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58a87728da34b67a63454017e7dc3fd', 'request_id': 'req-3dd7fb71-4083-454e-9a41-fe63bcf79a48'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.036916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58a87728da34b67a63454017e7dc3fd', 'request_id': 'req-3dd7fb71-4083-454e-9a41-fe63bcf79a48'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.056716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] The token cache is being invalidate because project d58a87728da34b67a63454017e7dc3fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:27.057558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.058382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.060871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1089/2145] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:26 2022] DELETE /identity/v3/projects/d58a87728da34b67a63454017e7dc3fd => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:27.064793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:27.067166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:27.155072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c371b53-5cee-48a5-b567-77108bde9ae8 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:27.162544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:27.163209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.163729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.164076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] PATH_INFO: `/v3/users/ff69128d173d4af784487e31b93b5746` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.165789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff69128d173d4af784487e31b93b5746)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:27.168081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:27.203923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:27.206045 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:27.258239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff69128d173d4af784487e31b93b5746'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.274844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff69128d173d4af784487e31b93b5746'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.275987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff69128d173d4af784487e31b93b5746'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.300087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff69128d173d4af784487e31b93b5746'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.301234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7fb71-4083-454e-9a41-fe63bcf79a48 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:27.308742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:27.309661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.310465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.311136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] PATH_INFO: `/v3/projects/6d252fa109fd4829b1d4337b0aa7fcf5` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.312769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d252fa109fd4829b1d4337b0aa7fcf5)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:27.313227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff69128d173d4af784487e31b93b5746'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.317684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:27.339714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1057/2146] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:27 2022] DELETE /identity/v3/users/ff69128d173d4af784487e31b93b5746 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:27.388251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:27.411753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d252fa109fd4829b1d4337b0aa7fcf5', 'request_id': 'req-adbbe37a-2243-4cdf-bd6d-426b0b258d46'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.420550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d252fa109fd4829b1d4337b0aa7fcf5', 'request_id': 'req-adbbe37a-2243-4cdf-bd6d-426b0b258d46'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.432621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d252fa109fd4829b1d4337b0aa7fcf5', 'request_id': 'req-adbbe37a-2243-4cdf-bd6d-426b0b258d46'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.455429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] The token cache is being invalidate because project 6d252fa109fd4829b1d4337b0aa7fcf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:27.455788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.456510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.461477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1090/2147] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:27 2022] DELETE /identity/v3/projects/6d252fa109fd4829b1d4337b0aa7fcf5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:27.462914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:27.465213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:27.564543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:27.601940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c2e20-689f-4dc4-92e1-8ef7afc56e8e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:27.720206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:27.720761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.721163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.721784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] PATH_INFO: `/v3/users/c216d1c6d8e5440d9ad80b598f57bdff` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.722841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=c216d1c6d8e5440d9ad80b598f57bdff)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:27.725104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:27.793359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c216d1c6d8e5440d9ad80b598f57bdff'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.794680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:27.796738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:27.817443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c216d1c6d8e5440d9ad80b598f57bdff'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.818156 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c216d1c6d8e5440d9ad80b598f57bdff'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.823380 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbe37a-2243-4cdf-bd6d-426b0b258d46 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:27.829644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:27.830461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.830824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.831223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] PATH_INFO: `/v3/projects/a281a2676a3045a9a4da9ad7e03032d0` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.832533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a281a2676a3045a9a4da9ad7e03032d0)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:27.834794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:27.836999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c216d1c6d8e5440d9ad80b598f57bdff'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.848833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-16aa59a4-a0cd-4385-82e3-6df4b99f97ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c216d1c6d8e5440d9ad80b598f57bdff'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.858996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1058/2148] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:27 2022] DELETE /identity/v3/users/c216d1c6d8e5440d9ad80b598f57bdff => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:27.903302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a281a2676a3045a9a4da9ad7e03032d0', 'request_id': 'req-40c7430d-5489-4e56-b777-317a1866f4cb'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.909931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a281a2676a3045a9a4da9ad7e03032d0', 'request_id': 'req-40c7430d-5489-4e56-b777-317a1866f4cb'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.917588 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a281a2676a3045a9a4da9ad7e03032d0', 'request_id': 'req-40c7430d-5489-4e56-b777-317a1866f4cb'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.939799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] The token cache is being invalidate because project a281a2676a3045a9a4da9ad7e03032d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:27.940138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.941391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-40c7430d-5489-4e56-b777-317a1866f4cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.943683 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1091/2149] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:27 2022] DELETE /identity/v3/projects/a281a2676a3045a9a4da9ad7e03032d0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:27.948574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e204621-9a01-456b-9425-19200027b91a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.949373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e204621-9a01-456b-9425-19200027b91a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.949373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e204621-9a01-456b-9425-19200027b91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.978896 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-5e204621-9a01-456b-9425-19200027b91a None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:27.980436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5e204621-9a01-456b-9425-19200027b91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.085931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1059/2150] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:28.094547 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b34fd-fe52-4a81-927a-c866bd234b6f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.094806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b34fd-fe52-4a81-927a-c866bd234b6f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.095074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b34fd-fe52-4a81-927a-c866bd234b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.124053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ff9b34fd-fe52-4a81-927a-c866bd234b6f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:28.125777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ff9b34fd-fe52-4a81-927a-c866bd234b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.147825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1092/2151] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:28.149198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e204621-9a01-456b-9425-19200027b91a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.164206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e204621-9a01-456b-9425-19200027b91a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.176285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.176921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.177206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.177478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.178340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:28.178646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.184508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.226123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1060/2152] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:28.231748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b34fd-fe52-4a81-927a-c866bd234b6f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.248278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b34fd-fe52-4a81-927a-c866bd234b6f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.262412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834307b-12b2-41d5-affa-7f7f882d4634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.263537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834307b-12b2-41d5-affa-7f7f882d4634 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.264042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834307b-12b2-41d5-affa-7f7f882d4634 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.264480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834307b-12b2-41d5-affa-7f7f882d4634 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.266701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834307b-12b2-41d5-affa-7f7f882d4634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.273558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834307b-12b2-41d5-affa-7f7f882d4634 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.277591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.280378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.301872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8a0d0-3834-4d5f-9c5a-8aed09a7e41e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.320049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.321657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.322370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.324080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] PATH_INFO: `/v3/projects/1ffaa7ac969848c993a521e63e0657b1` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.328332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ffaa7ac969848c993a521e63e0657b1)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.332228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.334988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:28.353262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1093/2153] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:28.358050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834307b-12b2-41d5-affa-7f7f882d4634 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.434229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ffaa7ac969848c993a521e63e0657b1', 'request_id': 'req-5e951760-1143-432e-aea4-e60fba613b72'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.441959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ffaa7ac969848c993a521e63e0657b1', 'request_id': 'req-5e951760-1143-432e-aea4-e60fba613b72'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.443393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834307b-12b2-41d5-affa-7f7f882d4634 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.453891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ffaa7ac969848c993a521e63e0657b1', 'request_id': 'req-5e951760-1143-432e-aea4-e60fba613b72'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.476510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] The token cache is being invalidate because project 1ffaa7ac969848c993a521e63e0657b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:28.476855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.477468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e951760-1143-432e-aea4-e60fba613b72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.481212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:28.481367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1061/2154] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:28 2022] DELETE /identity/v3/projects/1ffaa7ac969848c993a521e63e0657b1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:28.486591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6de234-65bc-4a24-b30d-8d7a17425171 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.486591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6de234-65bc-4a24-b30d-8d7a17425171 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.487020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6de234-65bc-4a24-b30d-8d7a17425171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.513589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-dc6de234-65bc-4a24-b30d-8d7a17425171 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:28.515172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dc6de234-65bc-4a24-b30d-8d7a17425171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.564681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.565928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.567556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.567831 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.568480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.570464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.625789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1062/2155] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:28.626533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee5b6a-dec6-4333-a54a-cf9cb4a9ede1 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.626533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee5b6a-dec6-4333-a54a-cf9cb4a9ede1 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.626730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee5b6a-dec6-4333-a54a-cf9cb4a9ede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.636562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1094/2156] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/users => generated 287 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:28.638044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.639546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.664623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de504d7-3a68-4949-8928-505a2ed8d9dc admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.666844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f2ee5b6a-dec6-4333-a54a-cf9cb4a9ede1 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:28.668364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f2ee5b6a-dec6-4333-a54a-cf9cb4a9ede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.683965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f39f-afdd-4810-9243-28a66887172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.684353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1063/2157] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:28.684664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f39f-afdd-4810-9243-28a66887172d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.684982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f39f-afdd-4810-9243-28a66887172d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.685256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9f39f-afdd-4810-9243-28a66887172d admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.685960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9f39f-afdd-4810-9243-28a66887172d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.687578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee5b6a-dec6-4333-a54a-cf9cb4a9ede1 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.698310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee5b6a-dec6-4333-a54a-cf9cb4a9ede1 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.704347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.705960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.706283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.706565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.707241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:28.707825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9f39f-afdd-4810-9243-28a66887172d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.709987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.712491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.733480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1095/2158] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:28.746262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1064/2159] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:28.746461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f39f-afdd-4810-9243-28a66887172d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.747437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.778388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9f39f-afdd-4810-9243-28a66887172d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.781763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88737d-ec2c-4e14-8074-f3c510646729 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.793361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490f24b-3aa4-45c0-926d-f183984fa5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.793920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490f24b-3aa4-45c0-926d-f183984fa5da admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.794196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490f24b-3aa4-45c0-926d-f183984fa5da admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.794467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490f24b-3aa4-45c0-926d-f183984fa5da admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.795131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490f24b-3aa4-45c0-926d-f183984fa5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.796802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a5bb-78d8-40ff-98d9-2d779efd1010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.797513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a5bb-78d8-40ff-98d9-2d779efd1010 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.797822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a5bb-78d8-40ff-98d9-2d779efd1010 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.798704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83a5bb-78d8-40ff-98d9-2d779efd1010 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.799050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83a5bb-78d8-40ff-98d9-2d779efd1010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.801890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83a5bb-78d8-40ff-98d9-2d779efd1010 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.811073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490f24b-3aa4-45c0-926d-f183984fa5da admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.845062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1096/2160] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:28.846002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490f24b-3aa4-45c0-926d-f183984fa5da admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.863928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490f24b-3aa4-45c0-926d-f183984fa5da admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.878029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1065/2161] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:28.878814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc1ea0-c75e-405e-90ab-f2a4a3b66ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.879554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc1ea0-c75e-405e-90ab-f2a4a3b66ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.879831 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc1ea0-c75e-405e-90ab-f2a4a3b66ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.880108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc1ea0-c75e-405e-90ab-f2a4a3b66ebc admin admin] PATH_INFO: `/v3/projects/e1388306728a45e79823fa7e3156c33a/users/ef7d925b638948649faedc030d14338f/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.881835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a5bb-78d8-40ff-98d9-2d779efd1010 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.903268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc1ea0-c75e-405e-90ab-f2a4a3b66ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1388306728a45e79823fa7e3156c33a, user_id=ef7d925b638948649faedc030d14338f, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.906306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc1ea0-c75e-405e-90ab-f2a4a3b66ebc admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.924455 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1097/2162] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:28 2022] PUT /identity/v3/projects/e1388306728a45e79823fa7e3156c33a/users/ef7d925b638948649faedc030d14338f/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:28.929715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeac79-94ab-41d8-a86b-3acb2b2ca531 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.930168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeac79-94ab-41d8-a86b-3acb2b2ca531 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.930533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeac79-94ab-41d8-a86b-3acb2b2ca531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.958768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-11eeac79-94ab-41d8-a86b-3acb2b2ca531 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:28.960336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-11eeac79-94ab-41d8-a86b-3acb2b2ca531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.997464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:29.039822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83a5bb-78d8-40ff-98d9-2d779efd1010 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.118345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1098/2163] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:29.126631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeac79-94ab-41d8-a86b-3acb2b2ca531 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.129178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-11eeac79-94ab-41d8-a86b-3acb2b2ca531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:29.181158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8149a1-9a30-4913-b1b4-3bdb88ce618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.181819 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8149a1-9a30-4913-b1b4-3bdb88ce618d admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.182100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8149a1-9a30-4913-b1b4-3bdb88ce618d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.182373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8149a1-9a30-4913-b1b4-3bdb88ce618d admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.183013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8149a1-9a30-4913-b1b4-3bdb88ce618d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.185234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8149a1-9a30-4913-b1b4-3bdb88ce618d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.232530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1066/2164] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/users => generated 287 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:29.234454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeac79-94ab-41d8-a86b-3acb2b2ca531 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.236474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8149a1-9a30-4913-b1b4-3bdb88ce618d admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.247778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6beb9f-a580-4e80-84d8-a7f8e75fb918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.249144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6beb9f-a580-4e80-84d8-a7f8e75fb918 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.249735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6beb9f-a580-4e80-84d8-a7f8e75fb918 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.250437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6beb9f-a580-4e80-84d8-a7f8e75fb918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.262838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8149a1-9a30-4913-b1b4-3bdb88ce618d admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.275411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5c04b-fd43-43c6-b946-6abe345f24d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.276196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5c04b-fd43-43c6-b946-6abe345f24d5 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.276485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5c04b-fd43-43c6-b946-6abe345f24d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.276756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5c04b-fd43-43c6-b946-6abe345f24d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.277725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5c04b-fd43-43c6-b946-6abe345f24d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.279627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6beb9f-a580-4e80-84d8-a7f8e75fb918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.283044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6beb9f-a580-4e80-84d8-a7f8e75fb918 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.290063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5c04b-fd43-43c6-b946-6abe345f24d5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.303731 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1067/2165] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:29.307967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5c04b-fd43-43c6-b946-6abe345f24d5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.318937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1099/2166] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:29.330100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5c04b-fd43-43c6-b946-6abe345f24d5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.342216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffdf00-a068-4002-836d-c96e6e085b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.342847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffdf00-a068-4002-836d-c96e6e085b94 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.343129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffdf00-a068-4002-836d-c96e6e085b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.343406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffdf00-a068-4002-836d-c96e6e085b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.344060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffdf00-a068-4002-836d-c96e6e085b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.345721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffdf00-a068-4002-836d-c96e6e085b94 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.368953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1068/2167] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:29.373328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6beb9f-a580-4e80-84d8-a7f8e75fb918 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.389794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6beb9f-a580-4e80-84d8-a7f8e75fb918 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.405212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.406020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.406375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.409599 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] PATH_INFO: `/v3/projects/8b08c4edb6ab4135b998ea5a2b684201/users/6f7536eebb424a9f941d6a549ea78b08/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.442160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:29.453374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b08c4edb6ab4135b998ea5a2b684201, user_id=6f7536eebb424a9f941d6a549ea78b08, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.462962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.497113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1100/2168] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:29 2022] PUT /identity/v3/projects/8b08c4edb6ab4135b998ea5a2b684201/users/6f7536eebb424a9f941d6a549ea78b08/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:29.511831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e0e8-0276-48a7-9f9f-fa8e21aec73f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.512495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e0e8-0276-48a7-9f9f-fa8e21aec73f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.512935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e0e8-0276-48a7-9f9f-fa8e21aec73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.554210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ed75e0e8-0276-48a7-9f9f-fa8e21aec73f None None] MFA Rules not processed for user `6f7536eebb424a9f941d6a549ea78b08`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:29.556194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ed75e0e8-0276-48a7-9f9f-fa8e21aec73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:29.669918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1069/2169] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:29.670712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.672733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:29.796136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a530e-83ad-4978-95c1-926333e04794 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.803235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d78c1e1b6e46e3ba6271e6743d3c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.805743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.805925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.806205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.817289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.819332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.833492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1101/2170] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:31.077063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d34a-d9b4-4c61-8396-025e9734c4c9 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.077447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d34a-d9b4-4c61-8396-025e9734c4c9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.077674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951d34a-d9b4-4c61-8396-025e9734c4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.119471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-0951d34a-d9b4-4c61-8396-025e9734c4c9 None None] MFA Rules not processed for user `ef7d925b638948649faedc030d14338f`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:31.120869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0951d34a-d9b4-4c61-8396-025e9734c4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:31.206440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:31.271886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1070/2171] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:31.282183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.290562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:31.361874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea727d-6c27-474d-8463-7df59265af5b service glance] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:31.367786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f9406-f6e4-4127-8a71-2c7976425494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:31.368209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9406-f6e4-4127-8a71-2c7976425494 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.368560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9406-f6e4-4127-8a71-2c7976425494 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.369916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9406-f6e4-4127-8a71-2c7976425494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.380828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f9406-f6e4-4127-8a71-2c7976425494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:31.382387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f9406-f6e4-4127-8a71-2c7976425494 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:31.392821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1102/2172] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:32.989928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a06bf-1116-466f-a2c2-cd9d63736206 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.990471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a06bf-1116-466f-a2c2-cd9d63736206 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.990471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9a06bf-1116-466f-a2c2-cd9d63736206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.016782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-fb9a06bf-1116-466f-a2c2-cd9d63736206 None None] MFA Rules not processed for user `ef7d925b638948649faedc030d14338f`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:33.017646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fb9a06bf-1116-466f-a2c2-cd9d63736206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.024937 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1071/2173] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:33.182128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afac2ff-7401-4da3-b061-2835442c59fb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.183007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afac2ff-7401-4da3-b061-2835442c59fb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.183288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afac2ff-7401-4da3-b061-2835442c59fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.212847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-8afac2ff-7401-4da3-b061-2835442c59fb None None] MFA Rules not processed for user `9cc388e0a26444f8a17601d762d58d07`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:33.214309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8afac2ff-7401-4da3-b061-2835442c59fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.319451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1103/2174] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:33.333802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a06bf-1116-466f-a2c2-cd9d63736206 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:33.335193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fb9a06bf-1116-466f-a2c2-cd9d63736206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.419885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9a06bf-1116-466f-a2c2-cd9d63736206 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:33.426992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:33.427645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.427926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.428198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.430119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.515077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:33.516700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:33.527799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1072/2175] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:43.314437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:43.522790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=67 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:47.833124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899cbc1-6d00-4601-9fec-ae6dab195aa3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.833631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899cbc1-6d00-4601-9fec-ae6dab195aa3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.833957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899cbc1-6d00-4601-9fec-ae6dab195aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.856050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-2899cbc1-6d00-4601-9fec-ae6dab195aa3 None None] MFA Rules not processed for user `ef7d925b638948649faedc030d14338f`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:47.857272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2899cbc1-6d00-4601-9fec-ae6dab195aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.864692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1104/2176] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:54.363241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:54.434917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa16ad-4624-4efc-a9cb-0ac01033b55a service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:54.440586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:54.441234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:54.441523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:54.441901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] PATH_INFO: `/v3/users/6f7536eebb424a9f941d6a549ea78b08` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:54.443057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f7536eebb424a9f941d6a549ea78b08)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:54.445185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:54.559759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f7536eebb424a9f941d6a549ea78b08'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:54.569257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f7536eebb424a9f941d6a549ea78b08'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:54.572363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f7536eebb424a9f941d6a549ea78b08'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:54.588838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f7536eebb424a9f941d6a549ea78b08'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:54.597203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f7536eebb424a9f941d6a549ea78b08'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:54.604456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1073/2177] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:54 2022] DELETE /identity/v3/users/6f7536eebb424a9f941d6a549ea78b08 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:54.609699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d45a87-8129-4945-8d1b-0244c35c9fab None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:54.610185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d45a87-8129-4945-8d1b-0244c35c9fab None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:54.610281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d45a87-8129-4945-8d1b-0244c35c9fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:54.630995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-35d45a87-8129-4945-8d1b-0244c35c9fab None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:54.632196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-35d45a87-8129-4945-8d1b-0244c35c9fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:54.707082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1105/2178] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:54.717333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:54.718768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:54.804049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87fcc4-319c-4d8f-9640-bb4ba855079c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:54.808334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b3883-070b-4a0b-ae7f-ba7785ab1f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:54.808951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3883-070b-4a0b-ae7f-ba7785ab1f75 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:54.809391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3883-070b-4a0b-ae7f-ba7785ab1f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:54.809705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b3883-070b-4a0b-ae7f-ba7785ab1f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:54.825751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3883-070b-4a0b-ae7f-ba7785ab1f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:54.827250 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b3883-070b-4a0b-ae7f-ba7785ab1f75 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:54.839633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1074/2179] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:55.143160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d45a87-8129-4945-8d1b-0244c35c9fab None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.144795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-35d45a87-8129-4945-8d1b-0244c35c9fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:55.157988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d45a87-8129-4945-8d1b-0244c35c9fab None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.162642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.163273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.163547 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.163820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] PATH_INFO: `/v3/projects/8b08c4edb6ab4135b998ea5a2b684201` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.164967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b08c4edb6ab4135b998ea5a2b684201)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.166915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.215742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b08c4edb6ab4135b998ea5a2b684201', 'request_id': 'req-9702ce58-91fb-4185-95c9-c244ab0147e3'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.221163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b08c4edb6ab4135b998ea5a2b684201', 'request_id': 'req-9702ce58-91fb-4185-95c9-c244ab0147e3'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.227379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b08c4edb6ab4135b998ea5a2b684201', 'request_id': 'req-9702ce58-91fb-4185-95c9-c244ab0147e3'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.240208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] The token cache is being invalidate because project 8b08c4edb6ab4135b998ea5a2b684201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:55.240517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.241178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9702ce58-91fb-4185-95c9-c244ab0147e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.242726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1106/2180] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:55 2022] DELETE /identity/v3/projects/8b08c4edb6ab4135b998ea5a2b684201 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:55.247628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41291d4b-1385-4e73-80bf-5445d8d0d944 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.247939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41291d4b-1385-4e73-80bf-5445d8d0d944 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.248218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41291d4b-1385-4e73-80bf-5445d8d0d944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.268353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-41291d4b-1385-4e73-80bf-5445d8d0d944 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:55.269552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-41291d4b-1385-4e73-80bf-5445d8d0d944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:55.290987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:55.335085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1075/2181] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:55.343938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577043-12f2-4012-8c7b-de892846c611 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.344258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577043-12f2-4012-8c7b-de892846c611 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.344565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577043-12f2-4012-8c7b-de892846c611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.364500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-7d577043-12f2-4012-8c7b-de892846c611 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:55.365751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7d577043-12f2-4012-8c7b-de892846c611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:55.373560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1107/2182] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:55.379368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41291d4b-1385-4e73-80bf-5445d8d0d944 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.389890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41291d4b-1385-4e73-80bf-5445d8d0d944 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.395352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.396022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.396321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.396556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.397893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:55.397893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.399444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.412603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1076/2183] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:55.417310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577043-12f2-4012-8c7b-de892846c611 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.434957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577043-12f2-4012-8c7b-de892846c611 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.439825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d3c87-c5e6-4411-aad7-d3bd575b7a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.440429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d3c87-c5e6-4411-aad7-d3bd575b7a46 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.440765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d3c87-c5e6-4411-aad7-d3bd575b7a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.441096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d3c87-c5e6-4411-aad7-d3bd575b7a46 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.441762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d3c87-c5e6-4411-aad7-d3bd575b7a46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.443672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d3c87-c5e6-4411-aad7-d3bd575b7a46 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.473108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1108/2184] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:55 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:55.477113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.487187 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e57a64-538e-41e2-8462-fbddd965a752 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.544542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758800a5-6c34-4c6b-88cc-3b3ede706496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.545275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758800a5-6c34-4c6b-88cc-3b3ede706496 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.545703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758800a5-6c34-4c6b-88cc-3b3ede706496 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.546059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758800a5-6c34-4c6b-88cc-3b3ede706496 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.546972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758800a5-6c34-4c6b-88cc-3b3ede706496 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.549427 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758800a5-6c34-4c6b-88cc-3b3ede706496 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.572729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1077/2185] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:55 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:55.576770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d3c87-c5e6-4411-aad7-d3bd575b7a46 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.587849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d3c87-c5e6-4411-aad7-d3bd575b7a46 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.592726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dddc3-9094-4718-a410-df2e54f2acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.593320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dddc3-9094-4718-a410-df2e54f2acff admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.593598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dddc3-9094-4718-a410-df2e54f2acff admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.593953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dddc3-9094-4718-a410-df2e54f2acff admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.594598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dddc3-9094-4718-a410-df2e54f2acff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.596273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584dddc3-9094-4718-a410-df2e54f2acff admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.606064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1109/2186] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:55.609821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758800a5-6c34-4c6b-88cc-3b3ede706496 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.619500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758800a5-6c34-4c6b-88cc-3b3ede706496 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.624110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237fbca-94ca-43ea-addf-23e178478537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.624691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237fbca-94ca-43ea-addf-23e178478537 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.624972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237fbca-94ca-43ea-addf-23e178478537 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.625243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237fbca-94ca-43ea-addf-23e178478537 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.625910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237fbca-94ca-43ea-addf-23e178478537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.628048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237fbca-94ca-43ea-addf-23e178478537 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.638435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1078/2187] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:55.642843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dddc3-9094-4718-a410-df2e54f2acff admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.653635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dddc3-9094-4718-a410-df2e54f2acff admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.658435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.659160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.659504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.659841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] PATH_INFO: `/v3/projects/af625e4a5d6144eba479918eaa8b049e/users/6906dddb121d4dea8a56fc3d58962caf/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.676875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af625e4a5d6144eba479918eaa8b049e, user_id=6906dddb121d4dea8a56fc3d58962caf, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.679778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.696040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1110/2188] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:55 2022] PUT /identity/v3/projects/af625e4a5d6144eba479918eaa8b049e/users/6906dddb121d4dea8a56fc3d58962caf/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:55.707431 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5e38-7e52-4e91-9ea3-2e868b65be81 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.707864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5e38-7e52-4e91-9ea3-2e868b65be81 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.708227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b5e38-7e52-4e91-9ea3-2e868b65be81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.739504 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f35b5e38-7e52-4e91-9ea3-2e868b65be81 None None] MFA Rules not processed for user `6906dddb121d4dea8a56fc3d58962caf`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:55.740881 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f35b5e38-7e52-4e91-9ea3-2e868b65be81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:55.834843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1079/2189] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:55.847182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.849701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:55.874777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:55.940572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8722f43-076b-4bb9-808b-5b6b61848519 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.945141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf5d6d-4165-44bc-9dda-7150fd0335a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.945999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf5d6d-4165-44bc-9dda-7150fd0335a5 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.946075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf5d6d-4165-44bc-9dda-7150fd0335a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.946396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf5d6d-4165-44bc-9dda-7150fd0335a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.957573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf5d6d-4165-44bc-9dda-7150fd0335a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.959208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf5d6d-4165-44bc-9dda-7150fd0335a5 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.968743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1111/2190] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:59.451157 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cc450-f3cd-4fd8-9946-0e352a7fdd0a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.451722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cc450-f3cd-4fd8-9946-0e352a7fdd0a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.451722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cc450-f3cd-4fd8-9946-0e352a7fdd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.471926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-9f0cc450-f3cd-4fd8-9946-0e352a7fdd0a None None] MFA Rules not processed for user `61b9ae7499154e5eab4f3c23d8bb6785`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:59.473112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9f0cc450-f3cd-4fd8-9946-0e352a7fdd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.484558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.530613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1080/2191] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:02.379228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadafe0-887f-431b-a9e4-defbd016a7e1 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.379819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadafe0-887f-431b-a9e4-defbd016a7e1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.380057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aadafe0-887f-431b-a9e4-defbd016a7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.399162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3aadafe0-887f-431b-a9e4-defbd016a7e1 None None] MFA Rules not processed for user `1e09b3554bb64828b857632d1de602d8`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:02.400375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3aadafe0-887f-431b-a9e4-defbd016a7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.456490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1112/2192] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:09.524881 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=17 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:10.958037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eab571-d22a-4218-b7bb-4110d82cb5d0 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.958629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eab571-d22a-4218-b7bb-4110d82cb5d0 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.958629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eab571-d22a-4218-b7bb-4110d82cb5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.977241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-94eab571-d22a-4218-b7bb-4110d82cb5d0 None None] MFA Rules not processed for user `a0e295fda2af4157b196c6130aa2d6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:10.978449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-94eab571-d22a-4218-b7bb-4110d82cb5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.038102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1081/2193] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:12.449697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=45 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.317334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadafe0-887f-431b-a9e4-defbd016a7e1 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.417096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aadafe0-887f-431b-a9e4-defbd016a7e1 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.422698 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.423786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.423786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.424032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] PATH_INFO: `/v3/users/6906dddb121d4dea8a56fc3d58962caf` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.425030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6906dddb121d4dea8a56fc3d58962caf)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.427281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.531291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6906dddb121d4dea8a56fc3d58962caf'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.543083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6906dddb121d4dea8a56fc3d58962caf'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.544735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6906dddb121d4dea8a56fc3d58962caf'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.569991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6906dddb121d4dea8a56fc3d58962caf'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.579306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6906dddb121d4dea8a56fc3d58962caf'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.589345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1113/2194] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:17 2022] DELETE /identity/v3/users/6906dddb121d4dea8a56fc3d58962caf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:17.591153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed817c-e40d-4420-9dee-75ac80933bd4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.591641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed817c-e40d-4420-9dee-75ac80933bd4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.591770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed817c-e40d-4420-9dee-75ac80933bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.615030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b4ed817c-e40d-4420-9dee-75ac80933bd4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:17.616353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b4ed817c-e40d-4420-9dee-75ac80933bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.685305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1082/2195] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:17.694785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.696746 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.756638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51554cf1-2efc-4edf-a441-e6376c302412 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.760379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d526ad1-009e-4e95-bdeb-6a5eea37ead2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.760990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d526ad1-009e-4e95-bdeb-6a5eea37ead2 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.761270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d526ad1-009e-4e95-bdeb-6a5eea37ead2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.761543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d526ad1-009e-4e95-bdeb-6a5eea37ead2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.772715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d526ad1-009e-4e95-bdeb-6a5eea37ead2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.774204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d526ad1-009e-4e95-bdeb-6a5eea37ead2 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.789821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1114/2196] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:18.348158 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed817c-e40d-4420-9dee-75ac80933bd4 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.350106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b4ed817c-e40d-4420-9dee-75ac80933bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.370972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed817c-e40d-4420-9dee-75ac80933bd4 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.377728 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.378727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.379202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.379662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] PATH_INFO: `/v3/projects/af625e4a5d6144eba479918eaa8b049e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.381022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] RBAC: Authorizing `identity:delete_project(project_id=af625e4a5d6144eba479918eaa8b049e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.384206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.457852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af625e4a5d6144eba479918eaa8b049e', 'request_id': 'req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.466379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af625e4a5d6144eba479918eaa8b049e', 'request_id': 'req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.474344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af625e4a5d6144eba479918eaa8b049e', 'request_id': 'req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.490214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] The token cache is being invalidate because project af625e4a5d6144eba479918eaa8b049e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:18.490515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.491700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-84df8228-edfd-46ff-b11d-6c9cd4f9b04f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.493705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1083/2197] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:18 2022] DELETE /identity/v3/projects/af625e4a5d6144eba479918eaa8b049e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:18.498459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c99087-92c6-4e1a-ada9-616ce006e67b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.499012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c99087-92c6-4e1a-ada9-616ce006e67b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.499516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c99087-92c6-4e1a-ada9-616ce006e67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.528356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-c7c99087-92c6-4e1a-ada9-616ce006e67b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.530169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c7c99087-92c6-4e1a-ada9-616ce006e67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.553937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:18.648871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1115/2198] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:18.657169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c640b17-de52-4028-8d80-1f5454db31ae None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.657719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c640b17-de52-4028-8d80-1f5454db31ae None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.658073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c640b17-de52-4028-8d80-1f5454db31ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.689680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7c640b17-de52-4028-8d80-1f5454db31ae None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.691794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7c640b17-de52-4028-8d80-1f5454db31ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.712220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1084/2199] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:18.712412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c99087-92c6-4e1a-ada9-616ce006e67b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.723595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c99087-92c6-4e1a-ada9-616ce006e67b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.729982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.730793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.731477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.731477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.732055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:18.732357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.734091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.761335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1116/2200] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:51:18.762778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c640b17-de52-4028-8d80-1f5454db31ae None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.775365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c640b17-de52-4028-8d80-1f5454db31ae None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.790960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9e8f-e588-4fb7-8920-e4298452ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.791898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9e8f-e588-4fb7-8920-e4298452ad3b admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.792375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9e8f-e588-4fb7-8920-e4298452ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.792888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9e8f-e588-4fb7-8920-e4298452ad3b admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.794173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb9e8f-e588-4fb7-8920-e4298452ad3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.797519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb9e8f-e588-4fb7-8920-e4298452ad3b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.844261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:18.863790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1085/2201] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:18.864811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.879284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06023d-8038-4bda-ae77-dd2524da27c3 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.938167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e851221-e243-4ae5-9aca-ebcccbf3255a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.938787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e851221-e243-4ae5-9aca-ebcccbf3255a admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.939073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e851221-e243-4ae5-9aca-ebcccbf3255a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.939349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e851221-e243-4ae5-9aca-ebcccbf3255a admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.939989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e851221-e243-4ae5-9aca-ebcccbf3255a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.942001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e851221-e243-4ae5-9aca-ebcccbf3255a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.965760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1117/2202] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:18.970247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9e8f-e588-4fb7-8920-e4298452ad3b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.980570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9e8f-e588-4fb7-8920-e4298452ad3b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.985651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644e5ab-3517-4f5d-987c-a71b8bc6b68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.986792 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644e5ab-3517-4f5d-987c-a71b8bc6b68a admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.987367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644e5ab-3517-4f5d-987c-a71b8bc6b68a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.987936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644e5ab-3517-4f5d-987c-a71b8bc6b68a admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.988718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644e5ab-3517-4f5d-987c-a71b8bc6b68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.990766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644e5ab-3517-4f5d-987c-a71b8bc6b68a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.005036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1086/2203] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:19.006436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e851221-e243-4ae5-9aca-ebcccbf3255a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.017865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e851221-e243-4ae5-9aca-ebcccbf3255a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.024590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af440e5-83d5-4679-bf01-235c6c924b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.025307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af440e5-83d5-4679-bf01-235c6c924b46 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.025862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af440e5-83d5-4679-bf01-235c6c924b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.026923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af440e5-83d5-4679-bf01-235c6c924b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.027758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af440e5-83d5-4679-bf01-235c6c924b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.029562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af440e5-83d5-4679-bf01-235c6c924b46 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.048554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1118/2204] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:19.050520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644e5ab-3517-4f5d-987c-a71b8bc6b68a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.064404 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644e5ab-3517-4f5d-987c-a71b8bc6b68a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.069337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.070050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.070345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.070613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] PATH_INFO: `/v3/projects/e96aeed18ac14c7bb29e471ca76c18f0/users/e53ea5c1d31640b58630579418296e9e/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.088933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96aeed18ac14c7bb29e471ca76c18f0, user_id=e53ea5c1d31640b58630579418296e9e, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.091848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.106090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1087/2205] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:19 2022] PUT /identity/v3/projects/e96aeed18ac14c7bb29e471ca76c18f0/users/e53ea5c1d31640b58630579418296e9e/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:19.117407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ca0e1-cc13-49ae-b6c6-96ebc8dbb0ad None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.117729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ca0e1-cc13-49ae-b6c6-96ebc8dbb0ad None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.118008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ca0e1-cc13-49ae-b6c6-96ebc8dbb0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.148384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-221ca0e1-cc13-49ae-b6c6-96ebc8dbb0ad None None] MFA Rules not processed for user `e53ea5c1d31640b58630579418296e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:19.149661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-221ca0e1-cc13-49ae-b6c6-96ebc8dbb0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.223143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1119/2206] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:19.235301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.237381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.323257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c420e-20f1-4cce-a615-dc910612fd12 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.330310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.331258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.331691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.332133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.345804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.347823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.370248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1088/2207] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:24.755040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0697c3-a521-4579-8489-a8a70eb4321b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.755597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0697c3-a521-4579-8489-a8a70eb4321b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.755597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0697c3-a521-4579-8489-a8a70eb4321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.775345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-7c0697c3-a521-4579-8489-a8a70eb4321b None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:24.776535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7c0697c3-a521-4579-8489-a8a70eb4321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:24.789176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:24.833728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1120/2208] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:51:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:24.878202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:24.879752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:24.960933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2ae57-2723-4803-9596-559febe14bbf service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:24.967443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133ba0e-9dc0-45b8-891c-3a126ae59e8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:24.968150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133ba0e-9dc0-45b8-891c-3a126ae59e8c service swift] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.968999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133ba0e-9dc0-45b8-891c-3a126ae59e8c service swift] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.969343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133ba0e-9dc0-45b8-891c-3a126ae59e8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.979305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133ba0e-9dc0-45b8-891c-3a126ae59e8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:24.981726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133ba0e-9dc0-45b8-891c-3a126ae59e8c service swift] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:24.990557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1089/2209] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 19:51:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:51:25.817196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d22f8-1968-4ae7-a8ed-0f97ce1f98a1 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.817737 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d22f8-1968-4ae7-a8ed-0f97ce1f98a1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.817847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d22f8-1968-4ae7-a8ed-0f97ce1f98a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.862708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a63d22f8-1968-4ae7-a8ed-0f97ce1f98a1 None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:25.863949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a63d22f8-1968-4ae7-a8ed-0f97ce1f98a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:25.880904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1121/2210] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:51:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:25.951097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133ba0e-9dc0-45b8-891c-3a126ae59e8c service swift] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.970006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133ba0e-9dc0-45b8-891c-3a126ae59e8c service swift] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.989323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63b7d9-441d-4f08-a8e8-a21f48eb2d0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.989978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b7d9-441d-4f08-a8e8-a21f48eb2d0f service swift] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.990261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b7d9-441d-4f08-a8e8-a21f48eb2d0f service swift] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.990537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63b7d9-441d-4f08-a8e8-a21f48eb2d0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:26.031775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63b7d9-441d-4f08-a8e8-a21f48eb2d0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:26.039674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63b7d9-441d-4f08-a8e8-a21f48eb2d0f service swift] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:26.073744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1090/2211] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 19:51:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:51:35.856166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=20 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:36.064292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=79,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:39.099175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d22f8-1968-4ae7-a8ed-0f97ce1f98a1 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.101381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a63d22f8-1968-4ae7-a8ed-0f97ce1f98a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.197152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d22f8-1968-4ae7-a8ed-0f97ce1f98a1 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.213988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391a2cf-fe3c-4a72-a4cb-521ba219cf9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.214652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a2cf-fe3c-4a72-a4cb-521ba219cf9d service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.214917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a2cf-fe3c-4a72-a4cb-521ba219cf9d service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.215187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391a2cf-fe3c-4a72-a4cb-521ba219cf9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.222678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d391a2cf-fe3c-4a72-a4cb-521ba219cf9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.384956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391a2cf-fe3c-4a72-a4cb-521ba219cf9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.386462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391a2cf-fe3c-4a72-a4cb-521ba219cf9d service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.414889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1122/2212] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:47.847210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86034761-1d5c-4f21-a5dd-14b1b6224897 None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:47.847873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86034761-1d5c-4f21-a5dd-14b1b6224897 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:47.847873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86034761-1d5c-4f21-a5dd-14b1b6224897 None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:47.849179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1091/2213] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:47.862599 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e3fe1-75ee-4baa-a158-20956b6988e5 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:47.863223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e3fe1-75ee-4baa-a158-20956b6988e5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:47.863223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e3fe1-75ee-4baa-a158-20956b6988e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:47.904657 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-162e3fe1-75ee-4baa-a158-20956b6988e5 None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:47.905834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-162e3fe1-75ee-4baa-a158-20956b6988e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:47.991449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1123/2214] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:51:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:48.453823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86034761-1d5c-4f21-a5dd-14b1b6224897 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:48.455600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-86034761-1d5c-4f21-a5dd-14b1b6224897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.514501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86034761-1d5c-4f21-a5dd-14b1b6224897 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:48.518109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff04b28-2112-450e-9b2c-af6de672c83d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d78c1e1b6e46e3ba6271e6743d3c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:48.518722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff04b28-2112-450e-9b2c-af6de672c83d service glance] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.519011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff04b28-2112-450e-9b2c-af6de672c83d service glance] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.519288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff04b28-2112-450e-9b2c-af6de672c83d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.529359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff04b28-2112-450e-9b2c-af6de672c83d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:48.530861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff04b28-2112-450e-9b2c-af6de672c83d service glance] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:48.542348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1092/2215] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:48.632629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b9495-86c6-4890-8a6f-0eb17466c232 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.632919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b9495-86c6-4890-8a6f-0eb17466c232 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.633196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b9495-86c6-4890-8a6f-0eb17466c232 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.634622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1124/2216] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:48.639550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348d48-fad3-436b-9ce6-6c2231d5e520 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.639858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348d48-fad3-436b-9ce6-6c2231d5e520 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.640133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94348d48-fad3-436b-9ce6-6c2231d5e520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.659846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-94348d48-fad3-436b-9ce6-6c2231d5e520 None None] MFA Rules not processed for user `9f77e57b2d96461c88e2fc33db22e22d`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:48.661050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-94348d48-fad3-436b-9ce6-6c2231d5e520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.667519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1093/2217] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:51:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:48.679901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b9495-86c6-4890-8a6f-0eb17466c232 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:48.691127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b9495-86c6-4890-8a6f-0eb17466c232 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:48.694615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207f051-df28-4b66-87c3-a79e1462ac30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:48.695196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207f051-df28-4b66-87c3-a79e1462ac30 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.695468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207f051-df28-4b66-87c3-a79e1462ac30 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.695738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207f051-df28-4b66-87c3-a79e1462ac30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.702637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207f051-df28-4b66-87c3-a79e1462ac30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:48.704186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207f051-df28-4b66-87c3-a79e1462ac30 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:48.711949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1125/2218] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:48.793064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1b56f-316b-4787-9b05-dbef7bd3b79b None None] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.793353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1b56f-316b-4787-9b05-dbef7bd3b79b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.793723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1b56f-316b-4787-9b05-dbef7bd3b79b None None] PATH_INFO: `/` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.795062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1094/2219] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 19:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:48.801125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61485dca-d32b-42fd-b61f-f9f18d7276ab None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.801485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61485dca-d32b-42fd-b61f-f9f18d7276ab None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.801797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61485dca-d32b-42fd-b61f-f9f18d7276ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.821939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-61485dca-d32b-42fd-b61f-f9f18d7276ab None None] MFA Rules not processed for user `2981bb0558d34329a061a5f08149bba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:48.823193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-61485dca-d32b-42fd-b61f-f9f18d7276ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.832849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1126/2220] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 19:51:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:48.843602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1b56f-316b-4787-9b05-dbef7bd3b79b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:48.845324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-17b1b56f-316b-4787-9b05-dbef7bd3b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.859049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1b56f-316b-4787-9b05-dbef7bd3b79b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:48.862765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a90b5-92a3-4a70-8346-adaddf5bd51a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:48.863357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a90b5-92a3-4a70-8346-adaddf5bd51a service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.863634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a90b5-92a3-4a70-8346-adaddf5bd51a service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.863904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a90b5-92a3-4a70-8346-adaddf5bd51a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.873981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a90b5-92a3-4a70-8346-adaddf5bd51a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:48.875419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a90b5-92a3-4a70-8346-adaddf5bd51a service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:48.885902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1095/2221] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:49.606655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca62b0-92c6-4d58-bdd8-5af67b374fdf None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.607162 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca62b0-92c6-4d58-bdd8-5af67b374fdf None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.607268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca62b0-92c6-4d58-bdd8-5af67b374fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.627145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-99ca62b0-92c6-4d58-bdd8-5af67b374fdf None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:49.628332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-99ca62b0-92c6-4d58-bdd8-5af67b374fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:49.636383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1127/2222] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 13 19:51:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:49.650167 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a90b5-92a3-4a70-8346-adaddf5bd51a service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:49.662102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a90b5-92a3-4a70-8346-adaddf5bd51a service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:49.667061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:49.667655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.667935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.668207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.676691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:49.678350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:49.691031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1096/2223] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 19:51:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:51:51.799781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab266f-3a3a-4448-a0b9-14ee1f461f08 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:51.800415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab266f-3a3a-4448-a0b9-14ee1f461f08 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:51.800415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab266f-3a3a-4448-a0b9-14ee1f461f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:51.807036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:51.809006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:51.823589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-61ab266f-3a3a-4448-a0b9-14ee1f461f08 None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:51.824807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-61ab266f-3a3a-4448-a0b9-14ee1f461f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:51.833378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1128/2224] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 19:51:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:51.846121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab266f-3a3a-4448-a0b9-14ee1f461f08 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:51.858082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab266f-3a3a-4448-a0b9-14ee1f461f08 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:51.863225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ea5b0-6491-4206-9cf4-8338ee5aacbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:51.863822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ea5b0-6491-4206-9cf4-8338ee5aacbf service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:51.864280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ea5b0-6491-4206-9cf4-8338ee5aacbf service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:51.864557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ea5b0-6491-4206-9cf4-8338ee5aacbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:51.870648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:51.874375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ea5b0-6491-4206-9cf4-8338ee5aacbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:51.876346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ea5b0-6491-4206-9cf4-8338ee5aacbf service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:51.885119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1129/2225] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 19:51:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:51:51.887414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cb1e0-270b-4bf6-9dc2-5e74d56ea113 service swift] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:51.897790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:51.899566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:51.899566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:51.899850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] PATH_INFO: `/v3/users/ef7d925b638948649faedc030d14338f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:51.902676 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7d925b638948649faedc030d14338f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:51.905019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:51.965414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7d925b638948649faedc030d14338f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:51.976801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7d925b638948649faedc030d14338f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:51.978313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7d925b638948649faedc030d14338f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:51.996274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7d925b638948649faedc030d14338f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:51.997860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:52.008352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6254b72e-8282-44c7-9aa8-d76b584ad7c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7d925b638948649faedc030d14338f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.023072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1097/2226] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:51 2022] DELETE /identity/v3/users/ef7d925b638948649faedc030d14338f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:52.279182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ea5b0-6491-4206-9cf4-8338ee5aacbf service swift] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:52.296064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ea5b0-6491-4206-9cf4-8338ee5aacbf service swift] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:52.440766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:52.440766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.440766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.440766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] PATH_INFO: `/v3/projects/e1388306728a45e79823fa7e3156c33a` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.442439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1388306728a45e79823fa7e3156c33a)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:52.444871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:52.500409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1388306728a45e79823fa7e3156c33a', 'request_id': 'req-5d3700f8-1505-4851-b199-cd88c0de9614'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.506047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1388306728a45e79823fa7e3156c33a', 'request_id': 'req-5d3700f8-1505-4851-b199-cd88c0de9614'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.512276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1388306728a45e79823fa7e3156c33a', 'request_id': 'req-5d3700f8-1505-4851-b199-cd88c0de9614'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.526488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] The token cache is being invalidate because project e1388306728a45e79823fa7e3156c33a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:52.526786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.527517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5d3700f8-1505-4851-b199-cd88c0de9614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.529072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1130/2227] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:52 2022] DELETE /identity/v3/projects/e1388306728a45e79823fa7e3156c33a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:52.535230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3751ba-739b-4187-b4f2-8654854c1c5f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.535518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3751ba-739b-4187-b4f2-8654854c1c5f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.535797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3751ba-739b-4187-b4f2-8654854c1c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.569205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-de3751ba-739b-4187-b4f2-8654854c1c5f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:52.569205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-de3751ba-739b-4187-b4f2-8654854c1c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:52.656729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1098/2228] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:52.665651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85632c7c-4984-4d2d-a22a-fe2479a503be None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.665948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85632c7c-4984-4d2d-a22a-fe2479a503be None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.666471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85632c7c-4984-4d2d-a22a-fe2479a503be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.694324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-85632c7c-4984-4d2d-a22a-fe2479a503be None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:52.695456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-85632c7c-4984-4d2d-a22a-fe2479a503be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:52.705388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1131/2229] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:52.709720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3751ba-739b-4187-b4f2-8654854c1c5f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:52.720091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3751ba-739b-4187-b4f2-8654854c1c5f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:52.724970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:52.725589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.725895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.726166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.726898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:52.727209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:52.728886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:52.745067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1099/2230] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:52.747618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85632c7c-4984-4d2d-a22a-fe2479a503be None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:52.762598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85632c7c-4984-4d2d-a22a-fe2479a503be None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:52.768806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59ec3d-b3b8-4db1-9009-cc38b01f1eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:52.769418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ec3d-b3b8-4db1-9009-cc38b01f1eff admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.769741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ec3d-b3b8-4db1-9009-cc38b01f1eff admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.770000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59ec3d-b3b8-4db1-9009-cc38b01f1eff admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.770649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59ec3d-b3b8-4db1-9009-cc38b01f1eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:52.772750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59ec3d-b3b8-4db1-9009-cc38b01f1eff admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:52.828101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1132/2231] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:52 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:52.834931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:52.850616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbd440-12da-42fe-bfe8-008006eb5415 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:52.939289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd15c8-7b47-46c0-9e7c-9bcddca8873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:52.940011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd15c8-7b47-46c0-9e7c-9bcddca8873b admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.940410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd15c8-7b47-46c0-9e7c-9bcddca8873b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.940757 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd15c8-7b47-46c0-9e7c-9bcddca8873b admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.941509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd15c8-7b47-46c0-9e7c-9bcddca8873b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:52.943634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd15c8-7b47-46c0-9e7c-9bcddca8873b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:52.977719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1100/2232] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:52 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:52.980368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59ec3d-b3b8-4db1-9009-cc38b01f1eff admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:52.992192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59ec3d-b3b8-4db1-9009-cc38b01f1eff admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:52.997155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0a0f-b2f9-442f-bbc8-33b8dce4b0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:52.997812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0a0f-b2f9-442f-bbc8-33b8dce4b0ed admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.998663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0a0f-b2f9-442f-bbc8-33b8dce4b0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.998919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0a0f-b2f9-442f-bbc8-33b8dce4b0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.999572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0a0f-b2f9-442f-bbc8-33b8dce4b0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.001179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0a0f-b2f9-442f-bbc8-33b8dce4b0ed admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.013708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1133/2233] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:53.019314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd15c8-7b47-46c0-9e7c-9bcddca8873b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.027800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd15c8-7b47-46c0-9e7c-9bcddca8873b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.032680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1167e-75f5-4cbb-bb10-4665391f0129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.033298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1167e-75f5-4cbb-bb10-4665391f0129 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.033581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1167e-75f5-4cbb-bb10-4665391f0129 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.033907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1167e-75f5-4cbb-bb10-4665391f0129 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.034592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1167e-75f5-4cbb-bb10-4665391f0129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.036217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1167e-75f5-4cbb-bb10-4665391f0129 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.049284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1101/2234] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:53.053497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0a0f-b2f9-442f-bbc8-33b8dce4b0ed admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.069575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0a0f-b2f9-442f-bbc8-33b8dce4b0ed admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.077976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.078739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.079065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.079340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] PATH_INFO: `/v3/projects/df360ab0b7594850b1e0b03b48e1f8ae/users/e5a2e785cf504586973630313c0b4607/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.098430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df360ab0b7594850b1e0b03b48e1f8ae, user_id=e5a2e785cf504586973630313c0b4607, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.101184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.115181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1134/2235] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:53 2022] PUT /identity/v3/projects/df360ab0b7594850b1e0b03b48e1f8ae/users/e5a2e785cf504586973630313c0b4607/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:53.120424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811f778-3b03-49ea-be6f-3210827f0453 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.120714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811f778-3b03-49ea-be6f-3210827f0453 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.120990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811f778-3b03-49ea-be6f-3210827f0453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.147122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-5811f778-3b03-49ea-be6f-3210827f0453 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:53.148628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5811f778-3b03-49ea-be6f-3210827f0453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:53.199952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:53.213234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1102/2236] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:53.223593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.225422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:53.255019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:53.293160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839588-2064-4353-a5a5-2893c15920b1 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.297919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa5d51-4950-4d6a-ad25-b65a2b877ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.298609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5d51-4950-4d6a-ad25-b65a2b877ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.298959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5d51-4950-4d6a-ad25-b65a2b877ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.299336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa5d51-4950-4d6a-ad25-b65a2b877ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.310050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa5d51-4950-4d6a-ad25-b65a2b877ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.312440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa5d51-4950-4d6a-ad25-b65a2b877ae0 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.324629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1135/2237] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:58.945122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdb41f-72b6-4c71-bbd0-960441da03d5 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:58.945690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdb41f-72b6-4c71-bbd0-960441da03d5 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:58.945690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdb41f-72b6-4c71-bbd0-960441da03d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.064414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-22fdb41f-72b6-4c71-bbd0-960441da03d5 None None] MFA Rules not processed for user `e5a2e785cf504586973630313c0b4607`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:59.070064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-22fdb41f-72b6-4c71-bbd0-960441da03d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:59.203910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1103/2238] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:51:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:59.208550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa5d51-4950-4d6a-ad25-b65a2b877ae0 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:59.219532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa5d51-4950-4d6a-ad25-b65a2b877ae0 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:59.224748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe13b6-ca4c-42db-bb96-a7d431ad1223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:59.225499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe13b6-ca4c-42db-bb96-a7d431ad1223 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:59.225886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe13b6-ca4c-42db-bb96-a7d431ad1223 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:59.226232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfe13b6-ca4c-42db-bb96-a7d431ad1223 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.226970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe13b6-ca4c-42db-bb96-a7d431ad1223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:59.228984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfe13b6-ca4c-42db-bb96-a7d431ad1223 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:59.280025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1136/2239] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:59 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:59.283445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdb41f-72b6-4c71-bbd0-960441da03d5 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:59.295088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdb41f-72b6-4c71-bbd0-960441da03d5 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:59.365018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34023e35-e51b-4c7c-9486-ddb5f4ef10bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:59.365660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34023e35-e51b-4c7c-9486-ddb5f4ef10bb admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:59.365944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34023e35-e51b-4c7c-9486-ddb5f4ef10bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:59.366212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34023e35-e51b-4c7c-9486-ddb5f4ef10bb admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.366843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34023e35-e51b-4c7c-9486-ddb5f4ef10bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:59.370257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34023e35-e51b-4c7c-9486-ddb5f4ef10bb admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:59.417112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1104/2240] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:59 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:59.418422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe13b6-ca4c-42db-bb96-a7d431ad1223 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:59.430786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfe13b6-ca4c-42db-bb96-a7d431ad1223 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:59.436397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd389d-8240-4e55-a8b4-e9fadaa6499d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:59.437010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd389d-8240-4e55-a8b4-e9fadaa6499d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:59.437287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd389d-8240-4e55-a8b4-e9fadaa6499d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:59.437558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd389d-8240-4e55-a8b4-e9fadaa6499d admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.438254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd389d-8240-4e55-a8b4-e9fadaa6499d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:59.440333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd389d-8240-4e55-a8b4-e9fadaa6499d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:59.455885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1137/2241] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:59.458105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34023e35-e51b-4c7c-9486-ddb5f4ef10bb admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:59.471442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34023e35-e51b-4c7c-9486-ddb5f4ef10bb admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:59.477445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54a0df-9dc1-43f8-bd39-72600116c0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:59.479549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a0df-9dc1-43f8-bd39-72600116c0c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:59.481809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a0df-9dc1-43f8-bd39-72600116c0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:59.482317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54a0df-9dc1-43f8-bd39-72600116c0c6 admin admin] PATH_INFO: `/v3/projects/49e24fb9671d4de28a59683209842d53/users/afc688e3405f41c9bd0fa1b313c8e7c4/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.500206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54a0df-9dc1-43f8-bd39-72600116c0c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e24fb9671d4de28a59683209842d53, user_id=afc688e3405f41c9bd0fa1b313c8e7c4, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:59.503090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54a0df-9dc1-43f8-bd39-72600116c0c6 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:59.525197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1105/2242] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:59 2022] PUT /identity/v3/projects/49e24fb9671d4de28a59683209842d53/users/afc688e3405f41c9bd0fa1b313c8e7c4/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:59.527692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd389d-8240-4e55-a8b4-e9fadaa6499d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:59.543942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd389d-8240-4e55-a8b4-e9fadaa6499d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:59.659027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db79949-7a24-4ec9-8b4a-4d1db47f8edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:59.659608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db79949-7a24-4ec9-8b4a-4d1db47f8edf admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:59.659886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db79949-7a24-4ec9-8b4a-4d1db47f8edf admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:59.660167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db79949-7a24-4ec9-8b4a-4d1db47f8edf admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.661006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db79949-7a24-4ec9-8b4a-4d1db47f8edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:59.664535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db79949-7a24-4ec9-8b4a-4d1db47f8edf admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:59.677785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1138/2243] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:59 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:59.682357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54a0df-9dc1-43f8-bd39-72600116c0c6 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:59.694754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54a0df-9dc1-43f8-bd39-72600116c0c6 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:59.700520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:59.701996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:59.702345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:59.702699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] PATH_INFO: `/v3/projects/49e24fb9671d4de28a59683209842d53/users/afc688e3405f41c9bd0fa1b313c8e7c4/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:59.704971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49e24fb9671d4de28a59683209842d53, user_id=afc688e3405f41c9bd0fa1b313c8e7c4, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:59.708996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:59.727611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1106/2244] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:59 2022] PUT /identity/v3/projects/49e24fb9671d4de28a59683209842d53/users/afc688e3405f41c9bd0fa1b313c8e7c4/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:52:05.224179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15b35-b0ea-4cec-ae1d-d34e283eab66 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:05.224708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15b35-b0ea-4cec-ae1d-d34e283eab66 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:05.224708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf15b35-b0ea-4cec-ae1d-d34e283eab66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:05.263033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bdf15b35-b0ea-4cec-ae1d-d34e283eab66 None None] MFA Rules not processed for user `afc688e3405f41c9bd0fa1b313c8e7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:05.264747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bdf15b35-b0ea-4cec-ae1d-d34e283eab66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:05.356462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1139/2245] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:05.387031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:05.389040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:05.452610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:05.490593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1e5641-2430-4509-a9a9-d2459c581560 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:05.497437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d0fdf-9896-48a0-a998-8e367680c91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:05.498334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0fdf-9896-48a0-a998-8e367680c91c service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:05.498788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0fdf-9896-48a0-a998-8e367680c91c service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:05.499215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0fdf-9896-48a0-a998-8e367680c91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:05.578774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d0fdf-9896-48a0-a998-8e367680c91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:05.580578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d0fdf-9896-48a0-a998-8e367680c91c service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:05.594887 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1107/2246] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:06.654504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfca30-72fe-40b0-9a64-c088ec82f0a1 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:06.655120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfca30-72fe-40b0-9a64-c088ec82f0a1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:06.655120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfca30-72fe-40b0-9a64-c088ec82f0a1 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:06.656577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1140/2247] 10.222.0.23 () {54 vars in 961 bytes} [Tue Sep 13 19:52:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:52:06.663789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1cf0-7fb6-4d02-8fd9-751b08b2de46 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:06.664744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1cf0-7fb6-4d02-8fd9-751b08b2de46 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:06.665364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc1cf0-7fb6-4d02-8fd9-751b08b2de46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:06.687283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-eefc1cf0-7fb6-4d02-8fd9-751b08b2de46 None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:06.688860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-eefc1cf0-7fb6-4d02-8fd9-751b08b2de46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:06.798365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1108/2248] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Sep 13 19:52:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:06.807160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfca30-72fe-40b0-9a64-c088ec82f0a1 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:06.824424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfca30-72fe-40b0-9a64-c088ec82f0a1 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:06.831567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:06.832375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:06.832830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:06.833266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:06.835907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:06.849600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:06.851241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:06.858099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:06.864613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1141/2249] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:06.915236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91740b-781c-413b-8afc-b5d7935e6f1b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:06.915532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91740b-781c-413b-8afc-b5d7935e6f1b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:06.915829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91740b-781c-413b-8afc-b5d7935e6f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:06.937785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-cb91740b-781c-413b-8afc-b5d7935e6f1b None None] MFA Rules not processed for user `e5a2e785cf504586973630313c0b4607`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:06.939102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cb91740b-781c-413b-8afc-b5d7935e6f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:06.946751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1109/2250] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:09.793121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:09.795191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:09.879015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97883383-b5cb-4031-bf59-46e3b93740d3 service placement] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:09.884089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:09.885063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:09.885459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:09.886422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] PATH_INFO: `/v3/users/e53ea5c1d31640b58630579418296e9e` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:09.887710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e53ea5c1d31640b58630579418296e9e)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:09.890262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:09.970459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e53ea5c1d31640b58630579418296e9e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:09.981995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e53ea5c1d31640b58630579418296e9e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:09.982814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e53ea5c1d31640b58630579418296e9e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:09.999722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e53ea5c1d31640b58630579418296e9e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:10.008458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e53ea5c1d31640b58630579418296e9e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:10.019511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1142/2251] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:09 2022] DELETE /identity/v3/users/e53ea5c1d31640b58630579418296e9e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:10.023981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522983a-5efe-482b-b389-9bce99b457a5 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.024461 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522983a-5efe-482b-b389-9bce99b457a5 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.025173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522983a-5efe-482b-b389-9bce99b457a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.044228 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7522983a-5efe-482b-b389-9bce99b457a5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:10.045430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7522983a-5efe-482b-b389-9bce99b457a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:10.106059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1110/2252] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:10.116245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:10.117835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:10.179829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af976e0-1c53-4dd1-97b7-7f0c1c6bc21d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:10.183800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb2072-3e14-4ed1-a40f-68b19c578133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:10.185117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb2072-3e14-4ed1-a40f-68b19c578133 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.185396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb2072-3e14-4ed1-a40f-68b19c578133 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.185707 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb2072-3e14-4ed1-a40f-68b19c578133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.194859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb2072-3e14-4ed1-a40f-68b19c578133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:10.196283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb2072-3e14-4ed1-a40f-68b19c578133 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:10.206324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1143/2253] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:10.562843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522983a-5efe-482b-b389-9bce99b457a5 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:10.564905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7522983a-5efe-482b-b389-9bce99b457a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:10.575729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:10.593721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522983a-5efe-482b-b389-9bce99b457a5 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:10.593721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:10.593721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.593721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.594565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] PATH_INFO: `/v3/projects/e96aeed18ac14c7bb29e471ca76c18f0` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.594565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96aeed18ac14c7bb29e471ca76c18f0)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:10.594565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:10.646255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96aeed18ac14c7bb29e471ca76c18f0', 'request_id': 'req-71aac1ef-be35-423d-b6f1-12f1c8abb7db'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:10.654441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96aeed18ac14c7bb29e471ca76c18f0', 'request_id': 'req-71aac1ef-be35-423d-b6f1-12f1c8abb7db'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:10.661116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96aeed18ac14c7bb29e471ca76c18f0', 'request_id': 'req-71aac1ef-be35-423d-b6f1-12f1c8abb7db'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:10.675730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] The token cache is being invalidate because project e96aeed18ac14c7bb29e471ca76c18f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:10.676021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:10.676664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-71aac1ef-be35-423d-b6f1-12f1c8abb7db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:10.678185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1111/2254] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:10 2022] DELETE /identity/v3/projects/e96aeed18ac14c7bb29e471ca76c18f0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:10.683041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b807de2-c66a-420e-9d60-04282f373c04 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.683327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b807de2-c66a-420e-9d60-04282f373c04 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.683600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b807de2-c66a-420e-9d60-04282f373c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.703065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5b807de2-c66a-420e-9d60-04282f373c04 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:10.704261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5b807de2-c66a-420e-9d60-04282f373c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:10.721692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:10.765081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1144/2255] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:10.772862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd68818-39c2-4008-ae51-56797e979bc6 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.773224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd68818-39c2-4008-ae51-56797e979bc6 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.773572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd68818-39c2-4008-ae51-56797e979bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.797746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1cd68818-39c2-4008-ae51-56797e979bc6 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:10.798889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1cd68818-39c2-4008-ae51-56797e979bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:10.806906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1112/2256] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:10.811111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b807de2-c66a-420e-9d60-04282f373c04 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:10.820982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b807de2-c66a-420e-9d60-04282f373c04 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:10.825570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:10.826179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.826452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.826718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.827388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:10.827686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:10.829310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:10.841491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1145/2257] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:52:10.845300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd68818-39c2-4008-ae51-56797e979bc6 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:10.854468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd68818-39c2-4008-ae51-56797e979bc6 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:10.859838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbee513-fa07-443d-80b3-cad6f00c5d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:10.860443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbee513-fa07-443d-80b3-cad6f00c5d14 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.860724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbee513-fa07-443d-80b3-cad6f00c5d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.861002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbee513-fa07-443d-80b3-cad6f00c5d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.861705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbee513-fa07-443d-80b3-cad6f00c5d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:10.863650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbee513-fa07-443d-80b3-cad6f00c5d14 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:10.889852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1113/2258] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:10 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:52:10.893964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:10.904482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0534a5c-de77-4ba3-9739-11bf38af1046 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:10.962678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ec52f-7549-4250-9f38-333f440b33e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:10.963324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec52f-7549-4250-9f38-333f440b33e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:10.963603 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec52f-7549-4250-9f38-333f440b33e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:10.963875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ec52f-7549-4250-9f38-333f440b33e2 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:10.964545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ec52f-7549-4250-9f38-333f440b33e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:10.967136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ec52f-7549-4250-9f38-333f440b33e2 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:10.988153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1146/2259] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:10 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:10.992534 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbee513-fa07-443d-80b3-cad6f00c5d14 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:11.002739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbee513-fa07-443d-80b3-cad6f00c5d14 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:11.007293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2462958-6f56-4706-8053-ae3cfe5b9293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:11.007885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2462958-6f56-4706-8053-ae3cfe5b9293 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.008165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2462958-6f56-4706-8053-ae3cfe5b9293 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.008439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2462958-6f56-4706-8053-ae3cfe5b9293 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.009074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2462958-6f56-4706-8053-ae3cfe5b9293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:11.010721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2462958-6f56-4706-8053-ae3cfe5b9293 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:11.023632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1114/2260] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:11.027715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ec52f-7549-4250-9f38-333f440b33e2 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:11.040662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ec52f-7549-4250-9f38-333f440b33e2 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:11.046997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e3f57-6868-4842-9f2e-3b76b2d09a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:11.047629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e3f57-6868-4842-9f2e-3b76b2d09a46 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.047960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e3f57-6868-4842-9f2e-3b76b2d09a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.048178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e3f57-6868-4842-9f2e-3b76b2d09a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.048825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e3f57-6868-4842-9f2e-3b76b2d09a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:11.050478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e3f57-6868-4842-9f2e-3b76b2d09a46 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:11.061747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1147/2261] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:11.066070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2462958-6f56-4706-8053-ae3cfe5b9293 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:11.076552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2462958-6f56-4706-8053-ae3cfe5b9293 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:11.081338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:11.082021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.082305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.082583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] PATH_INFO: `/v3/projects/d030d7bb8d92404e9083b3847fcb0487/users/1500cbd765c74fa488e678dda25f5e58/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.097638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d030d7bb8d92404e9083b3847fcb0487, user_id=1500cbd765c74fa488e678dda25f5e58, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:11.100313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:11.113092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1115/2262] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:11 2022] PUT /identity/v3/projects/d030d7bb8d92404e9083b3847fcb0487/users/1500cbd765c74fa488e678dda25f5e58/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:11.118215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af634-94eb-474d-9e6f-374302a09d1e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.118499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af634-94eb-474d-9e6f-374302a09d1e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.118828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16af634-94eb-474d-9e6f-374302a09d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.138931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d16af634-94eb-474d-9e6f-374302a09d1e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:11.140165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d16af634-94eb-474d-9e6f-374302a09d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:11.208040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1148/2263] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:11.218619 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:11.221080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:11.279537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e5ad5-2211-4e0d-bd9b-9793b3e7badd admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:11.283269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:11.283870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.284149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.284420 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.293254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:11.295243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:11.305173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1116/2264] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:52:11.961796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677de8-a778-42c4-84ec-94ae3e75d565 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.962694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677de8-a778-42c4-84ec-94ae3e75d565 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.963190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677de8-a778-42c4-84ec-94ae3e75d565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.994852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-db677de8-a778-42c4-84ec-94ae3e75d565 None None] MFA Rules not processed for user `1500cbd765c74fa488e678dda25f5e58`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:11.996172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-db677de8-a778-42c4-84ec-94ae3e75d565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:12.015123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:12.070195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1149/2265] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:12.081172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:12.082814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:12.119411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:12.148451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b781e-be8a-46b6-a443-16ec14a75b88 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:12.152069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213353c-7569-4c11-b337-3cc0471cedf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:12.152678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213353c-7569-4c11-b337-3cc0471cedf0 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:12.152992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213353c-7569-4c11-b337-3cc0471cedf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:12.153846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213353c-7569-4c11-b337-3cc0471cedf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:12.162272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213353c-7569-4c11-b337-3cc0471cedf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:12.163715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213353c-7569-4c11-b337-3cc0471cedf0 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:12.173463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1117/2266] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:13.554505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45148539-6171-4e08-bf85-b750b065b5fc None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.554963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45148539-6171-4e08-bf85-b750b065b5fc None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.555075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45148539-6171-4e08-bf85-b750b065b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.575280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-45148539-6171-4e08-bf85-b750b065b5fc None None] MFA Rules not processed for user `1500cbd765c74fa488e678dda25f5e58`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:13.576605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-45148539-6171-4e08-bf85-b750b065b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:13.583933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1150/2267] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:16.540189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa4f55-03d2-4517-8730-362a8c5bee73 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:16.540683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa4f55-03d2-4517-8730-362a8c5bee73 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:16.540767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa4f55-03d2-4517-8730-362a8c5bee73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:16.564035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f7aa4f55-03d2-4517-8730-362a8c5bee73 None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:16.565344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f7aa4f55-03d2-4517-8730-362a8c5bee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:16.624126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1118/2268] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 13 19:52:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:16.636254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45148539-6171-4e08-bf85-b750b065b5fc None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:16.638546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-45148539-6171-4e08-bf85-b750b065b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:16.707314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45148539-6171-4e08-bf85-b750b065b5fc None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:16.712672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:16.713272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:16.713549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:16.713850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:16.723812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:16.725420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:16.733951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1151/2269] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 19:52:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:52:26.616365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=39 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:26.730901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=54 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:42.865590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f36ed-2584-4b9f-8aff-9b87537c916e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:42.866159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f36ed-2584-4b9f-8aff-9b87537c916e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:42.866159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f36ed-2584-4b9f-8aff-9b87537c916e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:42.884793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b13f36ed-2584-4b9f-8aff-9b87537c916e None None] MFA Rules not processed for user `61b9ae7499154e5eab4f3c23d8bb6785`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:42.886030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b13f36ed-2584-4b9f-8aff-9b87537c916e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:42.940474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1119/2270] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:52:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:45.899378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.901127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:45.918238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568ad0b-316e-4db6-b3be-2a75c0ce1ad6 service swift] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.922530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89916b29-a5dc-4d12-a7f4-61ad1248dd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.923925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89916b29-a5dc-4d12-a7f4-61ad1248dd2b service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.924422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89916b29-a5dc-4d12-a7f4-61ad1248dd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.924898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89916b29-a5dc-4d12-a7f4-61ad1248dd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.927164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-89916b29-a5dc-4d12-a7f4-61ad1248dd2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:46.038746 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89916b29-a5dc-4d12-a7f4-61ad1248dd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.040668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89916b29-a5dc-4d12-a7f4-61ad1248dd2b service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.064500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1152/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:52.933863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=25 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:56.056144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=30 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:21.401944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca478d-5410-439f-b963-96fd6a6a60a2 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:21.402493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca478d-5410-439f-b963-96fd6a6a60a2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:21.402624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca478d-5410-439f-b963-96fd6a6a60a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:21.422365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e6ca478d-5410-439f-b963-96fd6a6a60a2 None None] MFA Rules not processed for user `a0e295fda2af4157b196c6130aa2d6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:21.423556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e6ca478d-5410-439f-b963-96fd6a6a60a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:21.477196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1120/2272] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:53:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:31.470524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=25 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:32.103823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec110-dee0-414a-8ea7-9ffd76873670 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:32.104323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec110-dee0-414a-8ea7-9ffd76873670 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:32.104427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ec110-dee0-414a-8ea7-9ffd76873670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:32.125192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-029ec110-dee0-414a-8ea7-9ffd76873670 None None] MFA Rules not processed for user `3177776f47ef4d3e937c2e8efb360e79`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:32.126428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-029ec110-dee0-414a-8ea7-9ffd76873670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:32.180076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1153/2273] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:38.303053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3278d-810d-4cd3-9a47-1c6f58f07559 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:38.303622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3278d-810d-4cd3-9a47-1c6f58f07559 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:38.303622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3278d-810d-4cd3-9a47-1c6f58f07559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:38.322580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7cb3278d-810d-4cd3-9a47-1c6f58f07559 None None] MFA Rules not processed for user `1500cbd765c74fa488e678dda25f5e58`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:38.323789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7cb3278d-810d-4cd3-9a47-1c6f58f07559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:38.331427 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1121/2274] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:53:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:42.173409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=25 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:48.317116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:54.745599 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1922b-eb52-4984-8ca1-fb26ff3ca4fc None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:54.746174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1922b-eb52-4984-8ca1-fb26ff3ca4fc None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:54.746174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1922b-eb52-4984-8ca1-fb26ff3ca4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:54.766172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-24c1922b-eb52-4984-8ca1-fb26ff3ca4fc None None] MFA Rules not processed for user `15069aa932aa473983c4102961b855d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:54.767333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-24c1922b-eb52-4984-8ca1-fb26ff3ca4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:54.822076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1154/2275] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 19:53:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:57.660605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3278d-810d-4cd3-9a47-1c6f58f07559 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:57.672693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3278d-810d-4cd3-9a47-1c6f58f07559 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:57.676469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:57.677184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:57.677539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:57.678086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:57.679930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:57.758739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:57.760185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:57.771358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1122/2276] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:00.149133 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1922b-eb52-4984-8ca1-fb26ff3ca4fc None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:00.159246 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1922b-eb52-4984-8ca1-fb26ff3ca4fc None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:00.162930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:00.163527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:00.163808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:00.164079 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:00.165872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:00.180408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:00.181866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:00.192798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1155/2277] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:07.766598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=34 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:10.187902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=31 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:10.995157 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:11.004422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec7627-2643-4abd-a567-dcfa335cc318 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:11.007959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:11.008547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:11.008826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:11.009094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:11.010831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:11.022853 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:11.024321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:11.034374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1123/2278] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:10 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:20.395761 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:20.405720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6544242-fbbf-407e-8c70-d23976022fc3 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:20.409364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:20.410004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:20.410283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:20.410553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:20.412302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:20.424030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:20.425475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:20.434539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1156/2279] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:21.029251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:30.430304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:30.897811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:30.899268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:30.914389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dd47b-2f02-4fc7-a449-82ea7af78266 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:30.919966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:30.920551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:30.920834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:30.921099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:30.922748 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:30.932636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:30.934241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:30.943371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1124/2280] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:40.939324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:44.479540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:44.481678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:44.497736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4915c8-94c6-4f5c-997e-ff9c57e18df1 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:44.503361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:44.503958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:44.504232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:44.504495 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:44.506226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:44.516878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:44.518491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:44.527695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1157/2281] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:54.523221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:57.026070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:57.036379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7922c73-9a51-4da4-ad50-077740298e05 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:57.040654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:57.041278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:57.041866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:57.042143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:57.044026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:57.056847 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:57.058621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:57.069811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1125/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:01.054850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:01.068148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c642880-f768-4c73-b288-7cfa3aae2a5c service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:01.084775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:01.085771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:01.086237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:01.087291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:01.089807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:01.106417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:01.107939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:01.122070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1158/2283] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:07.066065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:11.114984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:22.060089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:22.076284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ef7d-d849-48c2-ba15-1bde0b5e3cc4 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:22.083682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:22.084811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:22.085037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:22.085807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:22.087605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:22.103148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:22.104830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:22.118142 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1126/2284] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:32.112736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:34.350520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:34.363646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc098f-d64b-423a-a065-1ba0c4589dce service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:34.369317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:34.369959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:34.370243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:34.370509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:34.372288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:34.432101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:34.433757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:34.443789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1159/2285] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:44.439361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=28 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:06.234478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:06.302888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d179ca2-66f7-4e12-bb4a-a3f884ae335e service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:06.308720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:06.309321 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:06.309624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:06.309899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:06.311633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:06.383332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:06.385063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:06.395231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1127/2286] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:08.536662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:08.538373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:08.666280 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb81b1-0b43-4c5d-9b2f-565414a9e89d service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:08.671513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:08.672119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:08.672399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:08.672671 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:08.674454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:08.685696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:08.687264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:08.696907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1160/2287] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:16.390333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=56 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:18.692797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=59 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:57.113553 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:57.136930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020a725-d560-4a05-8869-6700dc9e3485 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:57.141599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:57.142331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:57.142615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:57.142898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:57.155518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:57.157012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:57.171072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1128/2288] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:59.249843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:59.265068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12d5e5-f45d-408c-884a-56dd454fd02b service placement] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:59.271404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82347288-d448-415f-85de-93e65af030e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:59.272275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82347288-d448-415f-85de-93e65af030e9 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:59.272718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82347288-d448-415f-85de-93e65af030e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:59.273149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82347288-d448-415f-85de-93e65af030e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:59.291597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82347288-d448-415f-85de-93e65af030e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:59.294034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82347288-d448-415f-85de-93e65af030e9 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:59.314089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1161/2289] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:01.344845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:01.347116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:01.406480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4743f-2757-4180-accd-1c49c340d90e service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:01.413078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b6310-6e56-4cb3-9045-1755a01becb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:01.414192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b6310-6e56-4cb3-9045-1755a01becb5 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:01.414674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b6310-6e56-4cb3-9045-1755a01becb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:01.415143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b6310-6e56-4cb3-9045-1755a01becb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:01.431726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b6310-6e56-4cb3-9045-1755a01becb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:01.434490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b6310-6e56-4cb3-9045-1755a01becb5 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:01.451062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1129/2290] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:09.304167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:11.443520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=25 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:11.560375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82347288-d448-415f-85de-93e65af030e9 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:11.578305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82347288-d448-415f-85de-93e65af030e9 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:11.587064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c9ff9-b85a-440c-9508-cbef0aad665d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:11.588070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c9ff9-b85a-440c-9508-cbef0aad665d service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:11.588556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c9ff9-b85a-440c-9508-cbef0aad665d service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:11.589029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c9ff9-b85a-440c-9508-cbef0aad665d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:11.603174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c9ff9-b85a-440c-9508-cbef0aad665d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:11.606114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c9ff9-b85a-440c-9508-cbef0aad665d service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:11.657188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1162/2291] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:21.187922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b6310-6e56-4cb3-9045-1755a01becb5 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:21.204591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b6310-6e56-4cb3-9045-1755a01becb5 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:21.210848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417f0e-11c1-4378-8958-1bc110451e21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:21.211695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417f0e-11c1-4378-8958-1bc110451e21 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:21.212077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417f0e-11c1-4378-8958-1bc110451e21 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:21.212477 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32417f0e-11c1-4378-8958-1bc110451e21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:21.222582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417f0e-11c1-4378-8958-1bc110451e21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:21.224256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32417f0e-11c1-4378-8958-1bc110451e21 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:21.259520 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1130/2292] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:59:21.653905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=13 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:31.255267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=13 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:07.231587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c9ff9-b85a-440c-9508-cbef0aad665d service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:07.252632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c9ff9-b85a-440c-9508-cbef0aad665d service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:07.256319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0982e1-b421-4977-b5c3-21cccbabf92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:07.257427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0982e1-b421-4977-b5c3-21cccbabf92a service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:07.257925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0982e1-b421-4977-b5c3-21cccbabf92a service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:07.258421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0982e1-b421-4977-b5c3-21cccbabf92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:07.270609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0982e1-b421-4977-b5c3-21cccbabf92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:07.272094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0982e1-b421-4977-b5c3-21cccbabf92a service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:07.282836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1163/2293] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:17.277662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:35.402038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417f0e-11c1-4378-8958-1bc110451e21 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:35.420755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32417f0e-11c1-4378-8958-1bc110451e21 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:35.431191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70015bf7-fef2-4bba-9bd3-55de3a96c26e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:35.432322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70015bf7-fef2-4bba-9bd3-55de3a96c26e service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:35.432823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70015bf7-fef2-4bba-9bd3-55de3a96c26e service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:35.433206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70015bf7-fef2-4bba-9bd3-55de3a96c26e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:35.444502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70015bf7-fef2-4bba-9bd3-55de3a96c26e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:35.447079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70015bf7-fef2-4bba-9bd3-55de3a96c26e service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:35.489276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1131/2294] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:45.486012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=13 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:01.687075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0982e1-b421-4977-b5c3-21cccbabf92a service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:01.702620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0982e1-b421-4977-b5c3-21cccbabf92a service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:01.708957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb02b0d-68b2-420e-a771-b743d6c98aaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:01.710063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb02b0d-68b2-420e-a771-b743d6c98aaf service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:01.710513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb02b0d-68b2-420e-a771-b743d6c98aaf service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:01.710972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb02b0d-68b2-420e-a771-b743d6c98aaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:01.723646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb02b0d-68b2-420e-a771-b743d6c98aaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:01.725109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb02b0d-68b2-420e-a771-b743d6c98aaf service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:01.763299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1164/2295] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:01.828750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70015bf7-fef2-4bba-9bd3-55de3a96c26e service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:01.845260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70015bf7-fef2-4bba-9bd3-55de3a96c26e service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:01.852101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc18c4-4b50-47c2-b9c1-0a35bd148084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:01.853150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc18c4-4b50-47c2-b9c1-0a35bd148084 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:01.853849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc18c4-4b50-47c2-b9c1-0a35bd148084 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:01.854203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc18c4-4b50-47c2-b9c1-0a35bd148084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:01.869888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc18c4-4b50-47c2-b9c1-0a35bd148084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:01.871471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc18c4-4b50-47c2-b9c1-0a35bd148084 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:01.907891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1132/2296] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:07.946294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb02b0d-68b2-420e-a771-b743d6c98aaf service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:07.968699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb02b0d-68b2-420e-a771-b743d6c98aaf service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:07.975738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:07.976630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:07.977149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:07.977764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:07.989827 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:07.992448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:08.008571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1165/2297] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:11.903857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=13 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:18.000697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=19 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:06.764176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc18c4-4b50-47c2-b9c1-0a35bd148084 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:06.781433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc18c4-4b50-47c2-b9c1-0a35bd148084 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:06.787447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:06.788073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:06.788363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:06.788636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:06.797685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:06.799384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:06.837117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1133/2298] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:02:12.381842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:12.384054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:12.410606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66afd71-3492-4eb3-82bd-11384a8c97dd service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:12.419289 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32d873-e2eb-4a3d-8fb1-f7fda542ac58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:12.420348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32d873-e2eb-4a3d-8fb1-f7fda542ac58 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:12.420842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32d873-e2eb-4a3d-8fb1-f7fda542ac58 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:12.421329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32d873-e2eb-4a3d-8fb1-f7fda542ac58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:12.437118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32d873-e2eb-4a3d-8fb1-f7fda542ac58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:12.440074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32d873-e2eb-4a3d-8fb1-f7fda542ac58 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:12.463292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1166/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:02:16.828858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=13 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:22.451335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:57.942023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:57.943606 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:58.014609 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bfab6-48de-446a-9445-0565f4c120bf service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:58.018869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:58.019103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:58.019390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:58.019667 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:58.021686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:58.106752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:58.108196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:58.119299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1134/2300] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:00.188356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32d873-e2eb-4a3d-8fb1-f7fda542ac58 service placement] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:00.197948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32d873-e2eb-4a3d-8fb1-f7fda542ac58 service placement] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:00.202471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:00.203088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:00.203364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:00.203635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:00.206126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:00.220956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:00.222403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:00.234145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1167/2301] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:08.114404 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=64 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:10.228777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:05:00.601552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:05:00.603886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:05:00.659913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043496cb-8356-4a30-9ffd-3090208e5f9d service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:05:00.664531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:05:00.665140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:05:00.666852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:05:00.667814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:05:00.670545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:05:00.755778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:05:00.757230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:05:00.767003 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1135/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:05:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:05:00.903621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:05:00.912978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509222-7dc9-4512-b854-0cf496e82989 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:05:00.916812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:05:00.917426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:05:00.917749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:05:00.918047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:05:00.919885 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:05:01.008053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:05:01.010154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:05:01.030135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1168/2303] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:05:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:05:02.490959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:05:02.499713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b467d-1309-4a1d-841b-bdf7cb3fa3b0 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:05:02.503314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:05:02.503900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:05:02.504174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:05:02.504441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:05:02.506276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:05:02.580909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:05:02.582555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:05:02.593495 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1136/2304] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:05:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:05:11.027417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=41 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:05:12.593828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=86 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:05:14.069216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:05:14.079171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2ce03-af90-4c7b-aedd-8b3474a91ba8 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:05:14.082794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:05:14.083391 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:05:14.083669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:05:14.083943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:05:14.085915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:05:14.095688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:05:14.097115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:05:14.105283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1169/2305] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:05:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:05:24.101796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:05:35.065480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:05:35.067707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:05:35.084729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31dbe70-e283-4d34-ac87-a082d19598db service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:05:35.090989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:05:35.091742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:05:35.092116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:05:35.092608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:05:35.095380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:05:35.177714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:05:35.179391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:05:35.194488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1137/2306] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:05:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:05:45.186681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=33 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:07:01.127662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:01.137939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75359068-1cdc-407a-98e8-4613a9270047 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:01.141722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:01.142790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:01.142790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:01.143131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:01.144911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:01.226951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:01.229646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:01.244926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1170/2307] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:07:01.295756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:01.305053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bda695-62c6-49b7-b928-c491fb4fc4fa service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:01.308703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:01.309306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:01.309584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:01.309883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:01.311723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:01.396948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:01.398708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:01.408212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1138/2308] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:07:04.977824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:05.006052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef3f44-3b7d-44fc-b632-489f4cd4be6a service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:05.020416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:05.021044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:05.021320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:05.021587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:05.024763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:05.044862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:05.046532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:05.058111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1171/2309] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:07:05.214914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:05.217146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:05.235526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510407-08dc-413f-93c1-335dafe88582 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:05.242197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:05.242836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:05.243120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:05.243393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:05.252670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:05.254330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:05.265391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1139/2310] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:07:09.240951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:09.260454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14450a8a-e520-4be3-a044-33ee74fda3b0 service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:09.264062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:09.264662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:09.264940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:09.265216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:09.267030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:09.380776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:09.382283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:09.398766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:07:09.402929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1172/2311] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:07:15.260296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=39 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:07:15.498324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:15.500093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:15.639245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1158af7-a1a4-4a6e-9a68-dc96a6df817f service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:15.646217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:15.646400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:15.646597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:15.646871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:15.649005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:15.660737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:15.662393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:15.672020 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1140/2312] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:07:25.667973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=59 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:03.907787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:03.946559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7343f-515d-4dfb-988d-e59e2ba889af service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:03.954396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:03.955026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:03.955331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:03.955742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:03.957578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:03.972738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:03.975764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:04.003218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1173/2313] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:08:13.983105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=13 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:58.145071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:58.163579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f7cc7-3e99-4b53-99b3-e44a667101c2 service placement] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:58.167336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe55518-c38b-4395-8517-7f377c957f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:58.167931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe55518-c38b-4395-8517-7f377c957f5a service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:58.168216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe55518-c38b-4395-8517-7f377c957f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:58.168489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe55518-c38b-4395-8517-7f377c957f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:58.179354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe55518-c38b-4395-8517-7f377c957f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:58.180798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe55518-c38b-4395-8517-7f377c957f5a service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:58.192346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1141/2314] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:09:08.186859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=10 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:10:16.199193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:10:16.200866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:10:16.212296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af639a26-cead-4694-9c78-fbedbe7c63a4 service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:10:16.215909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bac4f8-7dd3-4004-8278-fba70fdfd9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:10:16.216505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac4f8-7dd3-4004-8278-fba70fdfd9ef service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:10:16.216782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac4f8-7dd3-4004-8278-fba70fdfd9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:10:16.217057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bac4f8-7dd3-4004-8278-fba70fdfd9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:10:16.224555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bac4f8-7dd3-4004-8278-fba70fdfd9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:10:16.226024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bac4f8-7dd3-4004-8278-fba70fdfd9ef service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:10:16.234911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1174/2315] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:10:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:10:26.230416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:11:01.718694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe55518-c38b-4395-8517-7f377c957f5a service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:01.737411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe55518-c38b-4395-8517-7f377c957f5a service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:01.741024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e0e70-709c-4dad-8844-bb117bf596b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:01.741630 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0e70-709c-4dad-8844-bb117bf596b7 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:01.741907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0e70-709c-4dad-8844-bb117bf596b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:01.742176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e0e70-709c-4dad-8844-bb117bf596b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:01.750450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e0e70-709c-4dad-8844-bb117bf596b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:01.751854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e0e70-709c-4dad-8844-bb117bf596b7 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:01.779641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1142/2316] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:11:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:11:01.826795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bac4f8-7dd3-4004-8278-fba70fdfd9ef service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:01.836466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bac4f8-7dd3-4004-8278-fba70fdfd9ef service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:01.840074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae354d-f1db-45bf-9d5b-618d62feb2f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:01.841231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae354d-f1db-45bf-9d5b-618d62feb2f8 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:01.841511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae354d-f1db-45bf-9d5b-618d62feb2f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:01.841851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae354d-f1db-45bf-9d5b-618d62feb2f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:01.852197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae354d-f1db-45bf-9d5b-618d62feb2f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:01.853620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae354d-f1db-45bf-9d5b-618d62feb2f8 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:01.864415 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1175/2317] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:11:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:11:01.871468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e0e70-709c-4dad-8844-bb117bf596b7 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:01.879846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e0e70-709c-4dad-8844-bb117bf596b7 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:01.883426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd7509-70de-455a-aa56-70a1eb4b1460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:01.884018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7509-70de-455a-aa56-70a1eb4b1460 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:01.884299 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7509-70de-455a-aa56-70a1eb4b1460 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:01.884573 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cd7509-70de-455a-aa56-70a1eb4b1460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:01.894740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd7509-70de-455a-aa56-70a1eb4b1460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:01.896177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cd7509-70de-455a-aa56-70a1eb4b1460 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:01.905770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1143/2318] 10.222.0.23 () {60 vars in 1396 bytes} [Tue Sep 13 20:11:01 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:11:01.951815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae354d-f1db-45bf-9d5b-618d62feb2f8 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:01.960372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae354d-f1db-45bf-9d5b-618d62feb2f8 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:01.963940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850e9eb-a8a4-433e-b4eb-6d8bf9c2a92e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:01.964537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e9eb-a8a4-433e-b4eb-6d8bf9c2a92e service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:01.964817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e9eb-a8a4-433e-b4eb-6d8bf9c2a92e service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:01.965791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850e9eb-a8a4-433e-b4eb-6d8bf9c2a92e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:01.974950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850e9eb-a8a4-433e-b4eb-6d8bf9c2a92e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:01.976359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850e9eb-a8a4-433e-b4eb-6d8bf9c2a92e service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:02.003447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1176/2319] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:11:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:11:11.901052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=23 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:11:12.000252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=19 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:12:19.452735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd7509-70de-455a-aa56-70a1eb4b1460 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:12:19.464205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cd7509-70de-455a-aa56-70a1eb4b1460 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:12:19.469188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebcc48-9fab-4a35-a131-ed025ee5c743 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:12:19.469808 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebcc48-9fab-4a35-a131-ed025ee5c743 service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:12:19.470091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebcc48-9fab-4a35-a131-ed025ee5c743 service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:12:19.470363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebcc48-9fab-4a35-a131-ed025ee5c743 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:12:19.479019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebcc48-9fab-4a35-a131-ed025ee5c743 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:12:19.480618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebcc48-9fab-4a35-a131-ed025ee5c743 service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:12:19.490558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1144/2320] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:12:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:12:29.486503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:13:02.293259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850e9eb-a8a4-433e-b4eb-6d8bf9c2a92e service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:02.306375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850e9eb-a8a4-433e-b4eb-6d8bf9c2a92e service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:02.311837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c826078-59a8-40dd-af15-258ce5552822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:02.312435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c826078-59a8-40dd-af15-258ce5552822 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:02.312730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c826078-59a8-40dd-af15-258ce5552822 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:02.312964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c826078-59a8-40dd-af15-258ce5552822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:02.320879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c826078-59a8-40dd-af15-258ce5552822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:02.322513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c826078-59a8-40dd-af15-258ce5552822 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:02.351527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebcc48-9fab-4a35-a131-ed025ee5c743 service placement] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:02.362630 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebcc48-9fab-4a35-a131-ed025ee5c743 service placement] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:02.368070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1177/2321] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:13:02.368293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:02.368293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:02.368293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:02.368293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:02.381084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:02.382795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:02.395007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1145/2322] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:13:02.474869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c826078-59a8-40dd-af15-258ce5552822 service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:02.485463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c826078-59a8-40dd-af15-258ce5552822 service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:02.490931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:02.491638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:02.492098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:02.492487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:02.495412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:13:02.588091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:02.590138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:02.609336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1178/2323] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:13:03.474439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:03.475979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:13:03.500765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c3f0f-2878-4796-84c5-68f8297a3b15 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:03.504297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a602c-6216-4ba2-8d8d-69f53c6d1b4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:03.504885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a602c-6216-4ba2-8d8d-69f53c6d1b4b service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:03.505187 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a602c-6216-4ba2-8d8d-69f53c6d1b4b service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:03.505454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a602c-6216-4ba2-8d8d-69f53c6d1b4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:03.513919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a602c-6216-4ba2-8d8d-69f53c6d1b4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:03.515335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a602c-6216-4ba2-8d8d-69f53c6d1b4b service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:03.543259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1146/2324] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:13:03.585708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:03.595735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e88de-1aed-4a93-be50-10bb4d13fbe5 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:03.599316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7008632-802d-4471-ab43-cfa867271b94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:03.599904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008632-802d-4471-ab43-cfa867271b94 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:03.600179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008632-802d-4471-ab43-cfa867271b94 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:03.600448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7008632-802d-4471-ab43-cfa867271b94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:03.609436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7008632-802d-4471-ab43-cfa867271b94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:03.610908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7008632-802d-4471-ab43-cfa867271b94 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:03.640186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1179/2325] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:13:13.540034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=24 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:13:13.635501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=67 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:04.066748 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a602c-6216-4ba2-8d8d-69f53c6d1b4b service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:04.118987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a602c-6216-4ba2-8d8d-69f53c6d1b4b service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:04.122687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:04.123284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:04.123567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:04.123843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:04.125690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:04.206544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:04.207978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:04.219091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1147/2326] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:14.214414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=59 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:21.173456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7008632-802d-4471-ab43-cfa867271b94 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:21.226471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7008632-802d-4471-ab43-cfa867271b94 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:21.230708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:21.231233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:21.231508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:21.231778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:21.233648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:21.255205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:21.256614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:21.264646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1180/2327] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:31.261241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=31 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:17:04.411536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:17:04.420022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1882902f-084d-4878-a7f4-7cd95c8f76ad service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:17:04.423596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:17:04.424185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:17:04.424462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:17:04.424733 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:17:04.426579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:04.503380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:17:04.504804 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:17:04.526778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1148/2328] 10.222.0.23 () {60 vars in 1396 bytes} [Tue Sep 13 20:17:04 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:17:04.573971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:17:04.583502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016c704d-1950-4a7d-808b-7e854c4ba319 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:17:04.587816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:17:04.588417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:17:04.588694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:17:04.588966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:17:04.590841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:04.605399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:17:04.606860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:17:04.617715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1181/2329] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:17:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:17:04.736405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:17:04.745313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b16a3-0e43-4b8c-9f50-68c62b36c8d7 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:17:04.748877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:17:04.749465 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:17:04.749779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:17:04.750457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:17:04.752254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:04.822696 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:17:04.824350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:17:04.833221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1149/2330] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:17:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:17:14.612781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:17:14.829184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=74 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:17:26.439912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:17:26.441543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:26.578138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488dfbaf-43ee-42cc-8273-bfdaccad21a0 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:17:26.583309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:17:26.583914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:17:26.584195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:17:26.584471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:17:26.586333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:26.597987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:17:26.599872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:17:26.610124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1182/2331] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:17:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:17:36.606012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=63 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:02.558986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:02.560567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:02.590375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b723a-58ea-4952-b89a-8962c54a3728 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:02.593990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:02.594583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:02.594857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:02.595127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:02.596908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:02.668642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:02.670111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:02.679053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1150/2332] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:02.720706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:02.730188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63b1e3-7009-44e3-8789-9d98d6256748 service placement] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:02.733717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:02.734304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:02.734576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:02.734843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:02.736607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:02.810180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:02.811526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:02.820365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1183/2333] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:02.865314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:02.874287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b901b1-dcf3-48da-8136-62565fad5a3f service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:02.877883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:02.878482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:02.878762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:02.879036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:02.881072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:02.950878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:02.952356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:02.961484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1151/2334] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:04.958692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:04.958692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:04.997817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fbbe3-08fa-4f43-9df6-a155212a33c9 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:05.003587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:05.004454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:05.004735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:05.005022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:05.007203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:05.022216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:05.023261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:05.023883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:05.025447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:05.036039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1184/2335] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:05.036693 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:05.042391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8becb9-62e8-413f-97f3-1e1a8baf7b43 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:05.045536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5d93f-7dbe-40f6-8c2b-ba21dd61d430 service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:05.048362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:05.048969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:05.049211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:05.049351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:05.049540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:05.049822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:05.050102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:05.050375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:05.051498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:05.059987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:05.061411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:05.074575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1185/2336] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:05.114069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:05.131052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:05.132666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:05.142217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1152/2337] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:15.069138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=52 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:15.138028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=8 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:20:23.206317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:20:23.208044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:20:23.231935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a48e2f-bb3d-401f-9177-77c4ba8dd798 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:20:23.236157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214e8a5-c2ea-4821-82ad-8a4ab32b97a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:20:23.236762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214e8a5-c2ea-4821-82ad-8a4ab32b97a6 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:20:23.237040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214e8a5-c2ea-4821-82ad-8a4ab32b97a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:20:23.237315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214e8a5-c2ea-4821-82ad-8a4ab32b97a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:20:23.245403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214e8a5-c2ea-4821-82ad-8a4ab32b97a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:20:23.246919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214e8a5-c2ea-4821-82ad-8a4ab32b97a6 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:20:23.254957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1186/2338] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:20:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:20:33.251185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=10 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:21:05.154528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:21:05.174404 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e31c0-3724-4dc5-b025-e970d2f95db6 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:21:05.178820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62132c3-cc09-4380-938b-fc0b1551b271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:21:05.179423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62132c3-cc09-4380-938b-fc0b1551b271 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:21:05.179702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62132c3-cc09-4380-938b-fc0b1551b271 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:21:05.179973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62132c3-cc09-4380-938b-fc0b1551b271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:21:05.190840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62132c3-cc09-4380-938b-fc0b1551b271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:21:05.192305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62132c3-cc09-4380-938b-fc0b1551b271 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:21:05.203862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1153/2339] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:21:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:21:15.198454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=10 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:32.442901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214e8a5-c2ea-4821-82ad-8a4ab32b97a6 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:32.454547 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214e8a5-c2ea-4821-82ad-8a4ab32b97a6 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:32.459474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:32.460066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:32.460344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:32.460616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:32.469355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:32.470969 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:32.481347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1187/2340] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:42.476527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:04.004902 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62132c3-cc09-4380-938b-fc0b1551b271 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:04.014836 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62132c3-cc09-4380-938b-fc0b1551b271 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:04.018453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3cc82-37e1-4d85-a023-bdb4d61321c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:04.019045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3cc82-37e1-4d85-a023-bdb4d61321c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:04.019326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3cc82-37e1-4d85-a023-bdb4d61321c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:04.019601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3cc82-37e1-4d85-a023-bdb4d61321c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:04.028101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3cc82-37e1-4d85-a023-bdb4d61321c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:04.029524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3cc82-37e1-4d85-a023-bdb4d61321c0 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:04.056823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1154/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:23:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:23:05.423294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:05.425637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:05.456735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20dae6-edbf-4011-9aac-92ab9b096903 service placement] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:05.460825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:05.461294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:05.461575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:05.463607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:05.476210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:05.477674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:05.490819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1188/2342] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:23:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:23:05.498984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3cc82-37e1-4d85-a023-bdb4d61321c0 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:05.509888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3cc82-37e1-4d85-a023-bdb4d61321c0 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:05.514323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:05.514991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:05.515316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:05.515611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:05.526148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:05.527599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:05.537221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1155/2343] 10.222.0.23 () {60 vars in 1396 bytes} [Tue Sep 13 20:23:05 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:23:15.485977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=11 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:15.532541 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=19 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:04.848419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:04.850129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:04.862729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567da36-145c-47d4-bf51-438a5e6fce40 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:04.866337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa844ef-5e99-49d7-9932-4d5e8ab20505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:04.866935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa844ef-5e99-49d7-9932-4d5e8ab20505 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:04.867208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa844ef-5e99-49d7-9932-4d5e8ab20505 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:04.867474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa844ef-5e99-49d7-9932-4d5e8ab20505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:04.869150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9aa844ef-5e99-49d7-9932-4d5e8ab20505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:04.951612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa844ef-5e99-49d7-9932-4d5e8ab20505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:04.952957 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa844ef-5e99-49d7-9932-4d5e8ab20505 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:04.971774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1189/2344] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:07.984394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:07.986002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:08.001303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcd542-582b-44e8-bdbf-62253a039438 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:08.006277 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e915d86-6527-4117-b489-c78a4954b72b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:08.006882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915d86-6527-4117-b489-c78a4954b72b service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:08.007163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915d86-6527-4117-b489-c78a4954b72b service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:08.007435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e915d86-6527-4117-b489-c78a4954b72b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:08.009680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0e915d86-6527-4117-b489-c78a4954b72b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:08.021770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e915d86-6527-4117-b489-c78a4954b72b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:08.023382 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e915d86-6527-4117-b489-c78a4954b72b service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:08.034122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1156/2345] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:11.184632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe5a4-30b1-4101-a579-f92dec696205 None None] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:11.185207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe5a4-30b1-4101-a579-f92dec696205 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:11.185207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe5a4-30b1-4101-a579-f92dec696205 None None] PATH_INFO: `/` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:11.186563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1190/2346] 10.222.0.23 () {54 vars in 921 bytes} [Tue Sep 13 20:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 20:24:11.213184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43014253-55cb-4e92-b12f-f5b2b1ecccca None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:11.213754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43014253-55cb-4e92-b12f-f5b2b1ecccca None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:11.214048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43014253-55cb-4e92-b12f-f5b2b1ecccca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:11.247225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe5a4-30b1-4101-a579-f92dec696205 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:11.249048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c6efe5a4-30b1-4101-a579-f92dec696205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:11.267188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe5a4-30b1-4101-a579-f92dec696205 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:11.271966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:11.272864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:11.273169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:11.273421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] PATH_INFO: `/v3/users/1e09b3554bb64828b857632d1de602d8` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:11.274692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e09b3554bb64828b857632d1de602d8)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:11.276670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:11.279974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-43014253-55cb-4e92-b12f-f5b2b1ecccca None None] MFA Rules not processed for user `8957fc12ab74474f890b9188ba632fab`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:24:11.281274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-43014253-55cb-4e92-b12f-f5b2b1ecccca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:11.292116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1157/2347] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Sep 13 20:24:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:24:11.304099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43014253-55cb-4e92-b12f-f5b2b1ecccca None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:11.324222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43014253-55cb-4e92-b12f-f5b2b1ecccca None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:11.326065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e09b3554bb64828b857632d1de602d8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.329332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa97c5a-c4db-4353-92fe-ff2af9a2b11b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:11.329969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97c5a-c4db-4353-92fe-ff2af9a2b11b service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:11.330257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97c5a-c4db-4353-92fe-ff2af9a2b11b service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:11.330524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97c5a-c4db-4353-92fe-ff2af9a2b11b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:11.334620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e09b3554bb64828b857632d1de602d8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.335282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e09b3554bb64828b857632d1de602d8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.340091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa97c5a-c4db-4353-92fe-ff2af9a2b11b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:11.341700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa97c5a-c4db-4353-92fe-ff2af9a2b11b service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:11.348929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e09b3554bb64828b857632d1de602d8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.351601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1158/2348] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:11.355982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e09b3554bb64828b857632d1de602d8'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.362231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1191/2349] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:24:11 2022] DELETE /identity/v3/users/1e09b3554bb64828b857632d1de602d8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:24:11.537425 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa97c5a-c4db-4353-92fe-ff2af9a2b11b service placement] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:11.548563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa97c5a-c4db-4353-92fe-ff2af9a2b11b service placement] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:11.605965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:11.606536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:11.606818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:11.607088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] PATH_INFO: `/v3/projects/1f154705e31748d9b7b8455ee5f73b70` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:11.617129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f154705e31748d9b7b8455ee5f73b70)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:11.619670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:11.666967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f154705e31748d9b7b8455ee5f73b70', 'request_id': 'req-4829d840-19da-4db7-b084-b4aedb8e3a7e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.672526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f154705e31748d9b7b8455ee5f73b70', 'request_id': 'req-4829d840-19da-4db7-b084-b4aedb8e3a7e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.679007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f154705e31748d9b7b8455ee5f73b70', 'request_id': 'req-4829d840-19da-4db7-b084-b4aedb8e3a7e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.691503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] The token cache is being invalidate because project 1f154705e31748d9b7b8455ee5f73b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:24:11.691791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.692406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.693935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1159/2350] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:24:11 2022] DELETE /identity/v3/projects/1f154705e31748d9b7b8455ee5f73b70 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:24:11.698014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:11.699411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:11.712142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e239724-184a-4d29-b980-2a06aca82b1c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:11.749139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:11.773140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:11.773797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:11.774076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:11.774346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] PATH_INFO: `/v3/users/9cc388e0a26444f8a17601d762d58d07` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:11.787007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cc388e0a26444f8a17601d762d58d07)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:11.788964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:11.834237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cc388e0a26444f8a17601d762d58d07'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.841352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cc388e0a26444f8a17601d762d58d07'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.841987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cc388e0a26444f8a17601d762d58d07'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.854783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cc388e0a26444f8a17601d762d58d07'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.861655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cc388e0a26444f8a17601d762d58d07'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:11.867450 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1192/2351] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:24:11 2022] DELETE /identity/v3/users/9cc388e0a26444f8a17601d762d58d07 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:24:12.034750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:12.036185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:12.050206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829d840-19da-4db7-b084-b4aedb8e3a7e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:12.062917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:12.107718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:12.108338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:12.108616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:12.108885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] PATH_INFO: `/v3/projects/5596b9569d0d499dabb635403998c294` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:12.118936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5596b9569d0d499dabb635403998c294)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:12.120844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:12.166272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5596b9569d0d499dabb635403998c294', 'request_id': 'req-be6a9741-3119-4208-beb7-8d247bc4e682'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:12.171776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5596b9569d0d499dabb635403998c294', 'request_id': 'req-be6a9741-3119-4208-beb7-8d247bc4e682'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:12.178252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5596b9569d0d499dabb635403998c294', 'request_id': 'req-be6a9741-3119-4208-beb7-8d247bc4e682'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:12.190338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] The token cache is being invalidate because project 5596b9569d0d499dabb635403998c294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:24:12.190625 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:12.191243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:24:12.192699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1160/2352] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:24:12 2022] DELETE /identity/v3/projects/5596b9569d0d499dabb635403998c294 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:24:21.864012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=41,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:22.187467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:58.585725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:58.587610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:58.622837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:58.626510 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:58.710762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59698d9-0182-40e9-abe2-f52f0bc35c7a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:58.717681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:58.717681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:58.718023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:58.718359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:58.720311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:58.725337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a9741-3119-4208-beb7-8d247bc4e682 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:58.733351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:58.734001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:58.734410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:58.734559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:58.736360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] Could not find project: 1f154705e31748d9b7b8455ee5f73b70.: keystone.exception.ProjectNotFound: Could not find project: 1f154705e31748d9b7b8455ee5f73b70. Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 13 20:24:58.749664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:24:58.751140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:24:58.752515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application with Lock( Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 20:24:58.753779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 20:24:58.755019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 1f154705e31748d9b7b8455ee5f73b70. Sep 13 20:24:58.755019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: ERROR keystone.server.flask.application Sep 13 20:24:58.755019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1161/2353] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:58 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 129 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 13 20:24:58.755019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:58.755019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:58.807187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:58.808916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:58.818914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b740e31-bf20-4946-bc46-3e152e8d5fad service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:58.822834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:58.823457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:58.823739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:58.824550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:58.826515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:58.835480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1193/2354] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:58.956156 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:58.957635 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:58.977555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1162/2355] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:59.213419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:59.226254 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b150d31-6f37-4893-975f-1c15b2df89a6 service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:59.239984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:59.240633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:59.241034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:59.241308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:59.243109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:59.268352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:59.270022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:59.286779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21625db7-68d4-4f2b-83c8-f352f9472e72 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:59.292899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:59.293634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:59.293931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:59.294259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:59.296181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:59.313938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:59.342817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:59.347166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:59.367845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:59.372708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1194/2356] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:59.386129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:59.389803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:59.425789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1163/2357] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:24:59.508716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:59.520142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e1201-11cd-417b-90ba-fc4c31b0076d service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:59.526181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:59.526787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:59.527100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:59.527375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:59.529124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:59.545478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:59.546742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:59.557972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1195/2358] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:01.070131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:01.071890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:01.132292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:01.134160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:01.155054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a9db-6c6f-458a-b6eb-7cdc785e7456 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:01.160050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fab70e-cf80-44d7-baa6-b6312cdf42d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:01.160682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fab70e-cf80-44d7-baa6-b6312cdf42d7 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:01.161067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fab70e-cf80-44d7-baa6-b6312cdf42d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:01.161346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fab70e-cf80-44d7-baa6-b6312cdf42d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:01.163212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e4fab70e-cf80-44d7-baa6-b6312cdf42d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:01.232304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc195091-3dc6-453d-8ccd-3c5a96f72a3b service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:01.237160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:01.238376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:01.238914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:01.239292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:01.241213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:01.271318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fab70e-cf80-44d7-baa6-b6312cdf42d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:01.273231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fab70e-cf80-44d7-baa6-b6312cdf42d7 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:01.289048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1164/2359] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:01.332923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:01.334399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:01.345644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1196/2360] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:02.067263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09023c-1460-453a-808a-1c36badb4f5b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:02.067554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09023c-1460-453a-808a-1c36badb4f5b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:02.067829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09023c-1460-453a-808a-1c36badb4f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:02.097956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7f09023c-1460-453a-808a-1c36badb4f5b None None] MFA Rules not processed for user `61b9ae7499154e5eab4f3c23d8bb6785`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:02.099245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7f09023c-1460-453a-808a-1c36badb4f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:02.134014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=100 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:02.156042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1165/2361] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:02.515315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:02.517270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:02.530441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d414dcae-1fdb-4775-9a25-5c08616cca82 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:02.534674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:02.535402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:02.535868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:02.536237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:02.538210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:02.549071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:02.550709 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:02.561045 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1197/2362] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:02.965392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09023c-1460-453a-808a-1c36badb4f5b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:02.967372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7f09023c-1460-453a-808a-1c36badb4f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:03.062370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09023c-1460-453a-808a-1c36badb4f5b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:03.067720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:03.068277 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:03.068559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:03.068833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:03.070655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:03.086578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:03.088222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:03.098860 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1166/2363] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:06.320465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:06.326801 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:06.364362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952050c-88be-4858-9b5c-7e3632f79792 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:06.369161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:06.371233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:06.371509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:06.371783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] PATH_INFO: `/v3/users/15069aa932aa473983c4102961b855d5` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:06.372943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15069aa932aa473983c4102961b855d5)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:06.374984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:06.434845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:06.437070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:06.455907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15069aa932aa473983c4102961b855d5'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.460051 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae225a35-3497-4405-9b28-6151cc043454 service placement] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:06.466315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15069aa932aa473983c4102961b855d5'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.466554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:06.467340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:06.467614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15069aa932aa473983c4102961b855d5'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.467862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:06.468257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] PATH_INFO: `/v3/users/3177776f47ef4d3e937c2e8efb360e79` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:06.470160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3177776f47ef4d3e937c2e8efb360e79)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:06.472349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:06.497230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15069aa932aa473983c4102961b855d5'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.508627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:06.512273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15069aa932aa473983c4102961b855d5'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.521692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1198/2364] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:06 2022] DELETE /identity/v3/users/15069aa932aa473983c4102961b855d5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:06.539017 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3177776f47ef4d3e937c2e8efb360e79'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.548133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3177776f47ef4d3e937c2e8efb360e79'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.549113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3177776f47ef4d3e937c2e8efb360e79'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.563441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3177776f47ef4d3e937c2e8efb360e79'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.572260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3177776f47ef4d3e937c2e8efb360e79'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.578532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1167/2365] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:06 2022] DELETE /identity/v3/users/3177776f47ef4d3e937c2e8efb360e79 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:06.702066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:06.703543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:06.771830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:06.773637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:06.815188 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524185c-755f-42f8-b4f2-be9f50c8aae1 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:06.839728 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:06.864918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5ea6d-8184-4213-a880-791a94a5dc02 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:06.873297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:06.874124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:06.874411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:06.874688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] PATH_INFO: `/v3/projects/f03c01ff6ef24c59b65362c66ad1a1ea` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:06.876198 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f03c01ff6ef24c59b65362c66ad1a1ea)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:06.878293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:06.901411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:06.902800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:06.903082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:06.903356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] PATH_INFO: `/v3/projects/2981eddb8742492fac516851f761e2f9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:06.905802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2981eddb8742492fac516851f761e2f9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:06.908494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:06.965078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f03c01ff6ef24c59b65362c66ad1a1ea', 'request_id': 'req-36eae699-c5c0-4de8-9fd5-72a886b54769'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.976783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f03c01ff6ef24c59b65362c66ad1a1ea', 'request_id': 'req-36eae699-c5c0-4de8-9fd5-72a886b54769'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.980753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2981eddb8742492fac516851f761e2f9', 'request_id': 'req-119d2df0-e465-4bac-bd6c-c15a75daf99a'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.986618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f03c01ff6ef24c59b65362c66ad1a1ea', 'request_id': 'req-36eae699-c5c0-4de8-9fd5-72a886b54769'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:06.990068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2981eddb8742492fac516851f761e2f9', 'request_id': 'req-119d2df0-e465-4bac-bd6c-c15a75daf99a'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:07.003025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2981eddb8742492fac516851f761e2f9', 'request_id': 'req-119d2df0-e465-4bac-bd6c-c15a75daf99a'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:07.009626 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] The token cache is being invalidate because project f03c01ff6ef24c59b65362c66ad1a1ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:07.009925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:07.010798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-36eae699-c5c0-4de8-9fd5-72a886b54769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:07.013656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1168/2366] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:06 2022] DELETE /identity/v3/projects/f03c01ff6ef24c59b65362c66ad1a1ea => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:07.019189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc08a7-ad1f-465c-b1c5-04a3a5c18739 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.019617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc08a7-ad1f-465c-b1c5-04a3a5c18739 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.020027 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc08a7-ad1f-465c-b1c5-04a3a5c18739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.031826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] The token cache is being invalidate because project 2981eddb8742492fac516851f761e2f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:07.032116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:07.035317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-119d2df0-e465-4bac-bd6c-c15a75daf99a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:07.037089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1199/2367] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:06 2022] DELETE /identity/v3/projects/2981eddb8742492fac516851f761e2f9 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:07.042913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14c988-07cc-4671-b461-95235ee0460b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.043590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14c988-07cc-4671-b461-95235ee0460b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.043880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be14c988-07cc-4671-b461-95235ee0460b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.098374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-be14c988-07cc-4671-b461-95235ee0460b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:07.099597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-be14c988-07cc-4671-b461-95235ee0460b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:07.103694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2cfc08a7-ad1f-465c-b1c5-04a3a5c18739 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:07.104940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2cfc08a7-ad1f-465c-b1c5-04a3a5c18739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:07.105459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:07.170174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1200/2368] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:07.175199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1169/2369] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:07.178383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a44c-0638-4a3d-8dfa-31b802d3d336 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.178673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a44c-0638-4a3d-8dfa-31b802d3d336 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.178945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a44c-0638-4a3d-8dfa-31b802d3d336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.184283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc396419-1b4b-43f6-a715-444f4b0e9487 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.184658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc396419-1b4b-43f6-a715-444f4b0e9487 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.185010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc396419-1b4b-43f6-a715-444f4b0e9487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.200831 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-c012a44c-0638-4a3d-8dfa-31b802d3d336 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:07.202213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c012a44c-0638-4a3d-8dfa-31b802d3d336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:07.206038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-fc396419-1b4b-43f6-a715-444f4b0e9487 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:07.207211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fc396419-1b4b-43f6-a715-444f4b0e9487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:07.214772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1201/2370] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:07.215689 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012a44c-0638-4a3d-8dfa-31b802d3d336 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.218729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1170/2371] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:07.222883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc396419-1b4b-43f6-a715-444f4b0e9487 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.225216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012a44c-0638-4a3d-8dfa-31b802d3d336 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.231630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.232038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc396419-1b4b-43f6-a715-444f4b0e9487 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.232358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.232665 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.232902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.233546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:07.233889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.235477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.240042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.240662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.240949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.241823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.241974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:07.242221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.243910 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.248365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1202/2372] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:25:07.253220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.260767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1171/2373] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:25:07.262359 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.263038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dff3ba-e770-4a4e-93c4-6891a0410b7a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.267903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b776d-df93-48c7-9156-aa633dd407e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.268502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b776d-df93-48c7-9156-aa633dd407e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.268962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b776d-df93-48c7-9156-aa633dd407e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.269388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b776d-df93-48c7-9156-aa633dd407e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.270060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b776d-df93-48c7-9156-aa633dd407e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.272541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b776d-df93-48c7-9156-aa633dd407e6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.274131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bcf08-dc32-4b86-a8f0-9714530064c5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.289987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211455b-e47b-4926-8672-91f6f1c67b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.290675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211455b-e47b-4926-8672-91f6f1c67b53 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.290964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211455b-e47b-4926-8672-91f6f1c67b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.291246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211455b-e47b-4926-8672-91f6f1c67b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.292399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211455b-e47b-4926-8672-91f6f1c67b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.294501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211455b-e47b-4926-8672-91f6f1c67b53 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.327462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1203/2374] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:07.331355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b776d-df93-48c7-9156-aa633dd407e6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.334470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1172/2375] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:07.337544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211455b-e47b-4926-8672-91f6f1c67b53 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.359544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:07.418564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b776d-df93-48c7-9156-aa633dd407e6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.423829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa446e39-e7f0-45b1-8deb-541394be463e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.423994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211455b-e47b-4926-8672-91f6f1c67b53 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.424533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa446e39-e7f0-45b1-8deb-541394be463e admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.425313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa446e39-e7f0-45b1-8deb-541394be463e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.425313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa446e39-e7f0-45b1-8deb-541394be463e admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.425800 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa446e39-e7f0-45b1-8deb-541394be463e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.427926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa446e39-e7f0-45b1-8deb-541394be463e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.429165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc26d-2c0f-495b-930a-bb2a038213b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.429815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc26d-2c0f-495b-930a-bb2a038213b0 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.430100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc26d-2c0f-495b-930a-bb2a038213b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.430398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc26d-2c0f-495b-930a-bb2a038213b0 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.431005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cc26d-2c0f-495b-930a-bb2a038213b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.434077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cc26d-2c0f-495b-930a-bb2a038213b0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.458627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1204/2376] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:07.460229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1173/2377] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:07.462770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa446e39-e7f0-45b1-8deb-541394be463e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.464326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc26d-2c0f-495b-930a-bb2a038213b0 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.474806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc26d-2c0f-495b-930a-bb2a038213b0 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.476053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa446e39-e7f0-45b1-8deb-541394be463e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.480227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b22ff-63ce-425a-b6c8-d8bbae124dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.480832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b22ff-63ce-425a-b6c8-d8bbae124dfd admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.481116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b22ff-63ce-425a-b6c8-d8bbae124dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.481390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5b22ff-63ce-425a-b6c8-d8bbae124dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.481512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058f14f-71d8-4a1e-b00f-fc722f3a2231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.482091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b22ff-63ce-425a-b6c8-d8bbae124dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.482232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058f14f-71d8-4a1e-b00f-fc722f3a2231 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.482433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058f14f-71d8-4a1e-b00f-fc722f3a2231 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.482641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058f14f-71d8-4a1e-b00f-fc722f3a2231 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.483279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058f14f-71d8-4a1e-b00f-fc722f3a2231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.483687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5b22ff-63ce-425a-b6c8-d8bbae124dfd admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.485067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058f14f-71d8-4a1e-b00f-fc722f3a2231 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.497128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1174/2378] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:07.500874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1205/2379] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:07.502851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b22ff-63ce-425a-b6c8-d8bbae124dfd admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.506567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058f14f-71d8-4a1e-b00f-fc722f3a2231 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.522810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5b22ff-63ce-425a-b6c8-d8bbae124dfd admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.523589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058f14f-71d8-4a1e-b00f-fc722f3a2231 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.529433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f94b61-3d4f-4ab5-b851-e083781206aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.530048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca95e0a-fc77-442e-abde-c3c087147e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.530220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94b61-3d4f-4ab5-b851-e083781206aa admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.530332 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94b61-3d4f-4ab5-b851-e083781206aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.530597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f94b61-3d4f-4ab5-b851-e083781206aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.531496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f94b61-3d4f-4ab5-b851-e083781206aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.532143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca95e0a-fc77-442e-abde-c3c087147e12 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.532682 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca95e0a-fc77-442e-abde-c3c087147e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.533011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f94b61-3d4f-4ab5-b851-e083781206aa admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.534050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca95e0a-fc77-442e-abde-c3c087147e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.535674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca95e0a-fc77-442e-abde-c3c087147e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.537504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca95e0a-fc77-442e-abde-c3c087147e12 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.545709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1175/2380] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:07.548953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f94b61-3d4f-4ab5-b851-e083781206aa admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.553366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1206/2381] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:07.557177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca95e0a-fc77-442e-abde-c3c087147e12 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.564090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f94b61-3d4f-4ab5-b851-e083781206aa admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.571381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393f087-3c26-4aa9-adbc-78ecb2b06c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.572278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f087-3c26-4aa9-adbc-78ecb2b06c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.572562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca95e0a-fc77-442e-abde-c3c087147e12 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.574312 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f087-3c26-4aa9-adbc-78ecb2b06c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.574786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393f087-3c26-4aa9-adbc-78ecb2b06c3c admin admin] PATH_INFO: `/v3/projects/9563674062a542ce906d72c8c91e7fee/users/20b1cce654d84231abfc4062bf3c17e0/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.581437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e17709-0b51-4851-a4f9-51cc9595863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.582236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e17709-0b51-4851-a4f9-51cc9595863a admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.582654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e17709-0b51-4851-a4f9-51cc9595863a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.583073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e17709-0b51-4851-a4f9-51cc9595863a admin admin] PATH_INFO: `/v3/projects/c8f02786494b4a1a901c0d6240921180/users/17d37e2bf6324ee88f9f70da52cc63f4/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.597678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393f087-3c26-4aa9-adbc-78ecb2b06c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9563674062a542ce906d72c8c91e7fee, user_id=20b1cce654d84231abfc4062bf3c17e0, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.602043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393f087-3c26-4aa9-adbc-78ecb2b06c3c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.617382 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e17709-0b51-4851-a4f9-51cc9595863a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f02786494b4a1a901c0d6240921180, user_id=17d37e2bf6324ee88f9f70da52cc63f4, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.618296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1176/2382] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:07 2022] PUT /identity/v3/projects/9563674062a542ce906d72c8c91e7fee/users/20b1cce654d84231abfc4062bf3c17e0/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:07.621035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e17709-0b51-4851-a4f9-51cc9595863a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.629489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37634ecb-995e-420f-973c-1660e6c53c12 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.629811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37634ecb-995e-420f-973c-1660e6c53c12 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.630235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37634ecb-995e-420f-973c-1660e6c53c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.639941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1207/2383] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:07 2022] PUT /identity/v3/projects/c8f02786494b4a1a901c0d6240921180/users/17d37e2bf6324ee88f9f70da52cc63f4/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:07.651593 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e25fe-e98d-4a9a-9632-560953a131b5 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.652125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e25fe-e98d-4a9a-9632-560953a131b5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.652125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e25fe-e98d-4a9a-9632-560953a131b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.660031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:07.663488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-37634ecb-995e-420f-973c-1660e6c53c12 None None] MFA Rules not processed for user `20b1cce654d84231abfc4062bf3c17e0`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:07.664999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-37634ecb-995e-420f-973c-1660e6c53c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:07.682022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-297e25fe-e98d-4a9a-9632-560953a131b5 None None] MFA Rules not processed for user `17d37e2bf6324ee88f9f70da52cc63f4`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:07.683792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-297e25fe-e98d-4a9a-9632-560953a131b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:07.741410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1177/2384] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:07.746595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37634ecb-995e-420f-973c-1660e6c53c12 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.779896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1208/2385] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:07.794463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e25fe-e98d-4a9a-9632-560953a131b5 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.797269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-297e25fe-e98d-4a9a-9632-560953a131b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:07.816717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:07.838664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37634ecb-995e-420f-973c-1660e6c53c12 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.846011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc346278-1142-4f6d-8aae-b31073423777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.846824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc346278-1142-4f6d-8aae-b31073423777 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.847446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc346278-1142-4f6d-8aae-b31073423777 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.847724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc346278-1142-4f6d-8aae-b31073423777 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.848366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc346278-1142-4f6d-8aae-b31073423777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.850447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc346278-1142-4f6d-8aae-b31073423777 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:07.894390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1178/2386] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:07.895682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc346278-1142-4f6d-8aae-b31073423777 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:07.916476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e25fe-e98d-4a9a-9632-560953a131b5 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.987697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc346278-1142-4f6d-8aae-b31073423777 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:07.993285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401b294-c145-4e8d-91f0-5923079674fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:07.993958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401b294-c145-4e8d-91f0-5923079674fb admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:07.994243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401b294-c145-4e8d-91f0-5923079674fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:07.994523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401b294-c145-4e8d-91f0-5923079674fb admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:07.995979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401b294-c145-4e8d-91f0-5923079674fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:07.998029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401b294-c145-4e8d-91f0-5923079674fb admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:08.005399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c06f2-5679-47bf-9a8c-ce4c970f26fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:08.006108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c06f2-5679-47bf-9a8c-ce4c970f26fb service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:08.006340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c06f2-5679-47bf-9a8c-ce4c970f26fb service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:08.006613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c06f2-5679-47bf-9a8c-ce4c970f26fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:08.031914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1179/2387] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:07 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:08.035913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401b294-c145-4e8d-91f0-5923079674fb admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:08.036904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:08.048355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401b294-c145-4e8d-91f0-5923079674fb admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:08.054123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0c417-4255-4fc7-aa68-fe0b87a68189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:08.054690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0c417-4255-4fc7-aa68-fe0b87a68189 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:08.054967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0c417-4255-4fc7-aa68-fe0b87a68189 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:08.055237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0c417-4255-4fc7-aa68-fe0b87a68189 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:08.056186 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0c417-4255-4fc7-aa68-fe0b87a68189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:08.059197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0c417-4255-4fc7-aa68-fe0b87a68189 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:08.077023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1180/2388] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:08.078371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0c417-4255-4fc7-aa68-fe0b87a68189 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:08.084033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c06f2-5679-47bf-9a8c-ce4c970f26fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:08.085747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c06f2-5679-47bf-9a8c-ce4c970f26fb service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:08.092540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0c417-4255-4fc7-aa68-fe0b87a68189 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:08.097350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1209/2389] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:08.099067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be6c04-7af8-4e81-ab83-8723311b644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:08.099726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be6c04-7af8-4e81-ab83-8723311b644b admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:08.100030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be6c04-7af8-4e81-ab83-8723311b644b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:08.100394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be6c04-7af8-4e81-ab83-8723311b644b admin admin] PATH_INFO: `/v3/projects/ecda48200f9b4c359636d3a72a7808ef/users/62141ad58a394e1aa139221a835fe3ce/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:08.116848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be6c04-7af8-4e81-ab83-8723311b644b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecda48200f9b4c359636d3a72a7808ef, user_id=62141ad58a394e1aa139221a835fe3ce, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:08.119587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be6c04-7af8-4e81-ab83-8723311b644b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:08.138879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1181/2390] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:08 2022] PUT /identity/v3/projects/ecda48200f9b4c359636d3a72a7808ef/users/62141ad58a394e1aa139221a835fe3ce/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:08.150191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c06f2-5679-47bf-9a8c-ce4c970f26fb service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:08.162573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c06f2-5679-47bf-9a8c-ce4c970f26fb service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:08.246474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e762c6-a23f-4f81-8ed5-1f858a35bb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:08.247393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e762c6-a23f-4f81-8ed5-1f858a35bb7c admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:08.247875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e762c6-a23f-4f81-8ed5-1f858a35bb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:08.248422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e762c6-a23f-4f81-8ed5-1f858a35bb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:08.249565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e762c6-a23f-4f81-8ed5-1f858a35bb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:08.251614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e762c6-a23f-4f81-8ed5-1f858a35bb7c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:08.263907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1210/2391] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:08.268254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be6c04-7af8-4e81-ab83-8723311b644b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:08.281496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be6c04-7af8-4e81-ab83-8723311b644b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:08.288730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:08.288730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:08.288730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:08.289116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] PATH_INFO: `/v3/projects/ecda48200f9b4c359636d3a72a7808ef/users/62141ad58a394e1aa139221a835fe3ce/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:08.299737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecda48200f9b4c359636d3a72a7808ef, user_id=62141ad58a394e1aa139221a835fe3ce, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:08.303193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:08.326775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1182/2392] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:08 2022] PUT /identity/v3/projects/ecda48200f9b4c359636d3a72a7808ef/users/62141ad58a394e1aa139221a835fe3ce/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:08.328083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae695f19-b9fc-431a-8931-8d36a7c0b23f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:08.328924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae695f19-b9fc-431a-8931-8d36a7c0b23f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:08.329284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae695f19-b9fc-431a-8931-8d36a7c0b23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:08.362986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ae695f19-b9fc-431a-8931-8d36a7c0b23f None None] MFA Rules not processed for user `62141ad58a394e1aa139221a835fe3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:08.364250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ae695f19-b9fc-431a-8931-8d36a7c0b23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:08.481083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1211/2393] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:08.523168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:08.525029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:08.602459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b26d2-6161-42de-84a8-795e615e78af admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:08.607574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a1bed-6c13-4607-bac6-cd90b8bb4de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:08.608340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1bed-6c13-4607-bac6-cd90b8bb4de9 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:08.608976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1bed-6c13-4607-bac6-cd90b8bb4de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:08.609336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a1bed-6c13-4607-bac6-cd90b8bb4de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:08.637401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a1bed-6c13-4607-bac6-cd90b8bb4de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:08.649730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a1bed-6c13-4607-bac6-cd90b8bb4de9 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:08.664293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1183/2394] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:08.803625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae695f19-b9fc-431a-8931-8d36a7c0b23f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:08.824678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae695f19-b9fc-431a-8931-8d36a7c0b23f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:08.830037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4347672b-8faa-4a6f-97c9-0ab35000d0e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:08.830892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347672b-8faa-4a6f-97c9-0ab35000d0e6 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:08.831343 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347672b-8faa-4a6f-97c9-0ab35000d0e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:08.831938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347672b-8faa-4a6f-97c9-0ab35000d0e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:08.888663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:08.919439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347672b-8faa-4a6f-97c9-0ab35000d0e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:08.920923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4347672b-8faa-4a6f-97c9-0ab35000d0e6 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:08.930560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1212/2395] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:11.726720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a1bed-6c13-4607-bac6-cd90b8bb4de9 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:11.745063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:11.789688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a1bed-6c13-4607-bac6-cd90b8bb4de9 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:11.794234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:11.794854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:11.795135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:11.795408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] PATH_INFO: `/v3/users/20b1cce654d84231abfc4062bf3c17e0` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:11.796518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b1cce654d84231abfc4062bf3c17e0)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:11.798482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:11.842168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b1cce654d84231abfc4062bf3c17e0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:11.849771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b1cce654d84231abfc4062bf3c17e0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:11.850419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b1cce654d84231abfc4062bf3c17e0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:11.862605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b1cce654d84231abfc4062bf3c17e0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:11.869304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b1cce654d84231abfc4062bf3c17e0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:11.875762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1184/2396] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:11 2022] DELETE /identity/v3/users/20b1cce654d84231abfc4062bf3c17e0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:11.880267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd6346-3537-42ba-8a5e-b2422aed5049 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:11.880771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd6346-3537-42ba-8a5e-b2422aed5049 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:11.880883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbd6346-3537-42ba-8a5e-b2422aed5049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:11.899893 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4fbd6346-3537-42ba-8a5e-b2422aed5049 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:11.901074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4fbd6346-3537-42ba-8a5e-b2422aed5049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:11.958743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1213/2397] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:11.968396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:11.969803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:12.027116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07696-e49b-43ce-8659-18f4e41c597b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:12.030754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aefab6-d92e-41d3-bbf2-bc933cf97492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:12.031644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aefab6-d92e-41d3-bbf2-bc933cf97492 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:12.031921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aefab6-d92e-41d3-bbf2-bc933cf97492 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:12.032190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aefab6-d92e-41d3-bbf2-bc933cf97492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:12.040903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aefab6-d92e-41d3-bbf2-bc933cf97492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:12.042358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aefab6-d92e-41d3-bbf2-bc933cf97492 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:12.051691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1185/2398] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:12.320330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd6346-3537-42ba-8a5e-b2422aed5049 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:12.321808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4fbd6346-3537-42ba-8a5e-b2422aed5049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:12.338168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbd6346-3537-42ba-8a5e-b2422aed5049 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:12.343755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:12.344779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:12.345225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:12.345780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] PATH_INFO: `/v3/projects/9563674062a542ce906d72c8c91e7fee` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:12.348274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9563674062a542ce906d72c8c91e7fee)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:12.350766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:12.423794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9563674062a542ce906d72c8c91e7fee', 'request_id': 'req-80fffbad-513b-43b5-98d6-3a97d57a425f'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.430105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9563674062a542ce906d72c8c91e7fee', 'request_id': 'req-80fffbad-513b-43b5-98d6-3a97d57a425f'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.437308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9563674062a542ce906d72c8c91e7fee', 'request_id': 'req-80fffbad-513b-43b5-98d6-3a97d57a425f'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.452858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] The token cache is being invalidate because project 9563674062a542ce906d72c8c91e7fee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:12.453148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.453835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.455352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1214/2399] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:12 2022] DELETE /identity/v3/projects/9563674062a542ce906d72c8c91e7fee => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:12.459647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aefab6-d92e-41d3-bbf2-bc933cf97492 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:12.471616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aefab6-d92e-41d3-bbf2-bc933cf97492 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:12.510840 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:12.543271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:12.543915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:12.544260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:12.544536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] PATH_INFO: `/v3/users/62141ad58a394e1aa139221a835fe3ce` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:12.545726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62141ad58a394e1aa139221a835fe3ce)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:12.547791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:12.606524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62141ad58a394e1aa139221a835fe3ce'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.615556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62141ad58a394e1aa139221a835fe3ce'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.616192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62141ad58a394e1aa139221a835fe3ce'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.629385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62141ad58a394e1aa139221a835fe3ce'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.636330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-591ee024-88ca-4e98-8ec0-d25c7f283359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62141ad58a394e1aa139221a835fe3ce'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:12.642303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1186/2400] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:12 2022] DELETE /identity/v3/users/62141ad58a394e1aa139221a835fe3ce => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:12.914992 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:12.916490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:12.930928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fffbad-513b-43b5-98d6-3a97d57a425f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:12.988544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:12.989175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:12.989452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:12.989764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] PATH_INFO: `/v3/projects/ecda48200f9b4c359636d3a72a7808ef` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:12.990868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecda48200f9b4c359636d3a72a7808ef)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:12.992787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:12.995266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:13.041408 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecda48200f9b4c359636d3a72a7808ef', 'request_id': 'req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:13.046947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecda48200f9b4c359636d3a72a7808ef', 'request_id': 'req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:13.053052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecda48200f9b4c359636d3a72a7808ef', 'request_id': 'req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:13.064972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] The token cache is being invalidate because project ecda48200f9b4c359636d3a72a7808ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:13.065262 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:13.065899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b6ec46bb-0d97-4d0e-8cd4-24fe762ef2aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:13.067372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1215/2401] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:12 2022] DELETE /identity/v3/projects/ecda48200f9b4c359636d3a72a7808ef => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:13.072212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89d53c-de0e-45df-82a1-7c0b603aa08e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.072730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89d53c-de0e-45df-82a1-7c0b603aa08e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.072865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89d53c-de0e-45df-82a1-7c0b603aa08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.093317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-4c89d53c-de0e-45df-82a1-7c0b603aa08e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:13.094546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4c89d53c-de0e-45df-82a1-7c0b603aa08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:13.160256 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1187/2402] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:13.163401 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18da46-281d-4a69-8c93-0a9a1cd8b536 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.163688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18da46-281d-4a69-8c93-0a9a1cd8b536 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.163968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18da46-281d-4a69-8c93-0a9a1cd8b536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.183985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-6b18da46-281d-4a69-8c93-0a9a1cd8b536 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:13.185185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6b18da46-281d-4a69-8c93-0a9a1cd8b536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:13.193055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1216/2403] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:13.197287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89d53c-de0e-45df-82a1-7c0b603aa08e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.207433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89d53c-de0e-45df-82a1-7c0b603aa08e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.212345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.213201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.213741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.214971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.215815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:13.215815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.217433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.229686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1188/2404] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:25:13.234083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18da46-281d-4a69-8c93-0a9a1cd8b536 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.243127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18da46-281d-4a69-8c93-0a9a1cd8b536 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.248089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbb981-6e64-4d19-a656-5a4212b27400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.248687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb981-6e64-4d19-a656-5a4212b27400 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.248961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb981-6e64-4d19-a656-5a4212b27400 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.249228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcbb981-6e64-4d19-a656-5a4212b27400 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.249871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbb981-6e64-4d19-a656-5a4212b27400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.251774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcbb981-6e64-4d19-a656-5a4212b27400 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.281390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1217/2405] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:13 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:13.285500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.302360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732f1ff-8f4f-465b-8d9f-bf23e80ee0f5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.347629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:13.364716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfdfe6-ea79-47e8-be9b-ef39dbaa1e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.365457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfdfe6-ea79-47e8-be9b-ef39dbaa1e35 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.365795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfdfe6-ea79-47e8-be9b-ef39dbaa1e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.366563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfdfe6-ea79-47e8-be9b-ef39dbaa1e35 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.366780 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfdfe6-ea79-47e8-be9b-ef39dbaa1e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.368714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfdfe6-ea79-47e8-be9b-ef39dbaa1e35 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.395481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1189/2406] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:13 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:13.396787 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbb981-6e64-4d19-a656-5a4212b27400 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.416595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcbb981-6e64-4d19-a656-5a4212b27400 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.422052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783b09e-a5b9-4503-8af4-0a1a728a1f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.422674 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783b09e-a5b9-4503-8af4-0a1a728a1f72 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.422952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783b09e-a5b9-4503-8af4-0a1a728a1f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.423221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783b09e-a5b9-4503-8af4-0a1a728a1f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.423872 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783b09e-a5b9-4503-8af4-0a1a728a1f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.425477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783b09e-a5b9-4503-8af4-0a1a728a1f72 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.440701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1218/2407] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:13.441654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfdfe6-ea79-47e8-be9b-ef39dbaa1e35 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.452936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfdfe6-ea79-47e8-be9b-ef39dbaa1e35 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.459502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbb37be-b6ed-49f1-8493-831609a6c9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.460140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb37be-b6ed-49f1-8493-831609a6c9fe admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.460423 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb37be-b6ed-49f1-8493-831609a6c9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.460741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbb37be-b6ed-49f1-8493-831609a6c9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.470733 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb37be-b6ed-49f1-8493-831609a6c9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.472447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbb37be-b6ed-49f1-8493-831609a6c9fe admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.484473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1190/2408] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:13.488732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783b09e-a5b9-4503-8af4-0a1a728a1f72 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.500183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783b09e-a5b9-4503-8af4-0a1a728a1f72 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.505004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d12b62-c990-4101-9734-c23a2d61101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.505687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d12b62-c990-4101-9734-c23a2d61101a admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.506004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d12b62-c990-4101-9734-c23a2d61101a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.506234 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d12b62-c990-4101-9734-c23a2d61101a admin admin] PATH_INFO: `/v3/projects/894e75d31e644dbbbfbfe9730ccaecd0/users/da93146b1fc14bf4a9595bb6576fbf98/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.522731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d12b62-c990-4101-9734-c23a2d61101a admin admin] RBAC: Authorizing `identity:create_grant(project_id=894e75d31e644dbbbfbfe9730ccaecd0, user_id=da93146b1fc14bf4a9595bb6576fbf98, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.526178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d12b62-c990-4101-9734-c23a2d61101a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.542505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1219/2409] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:13 2022] PUT /identity/v3/projects/894e75d31e644dbbbfbfe9730ccaecd0/users/da93146b1fc14bf4a9595bb6576fbf98/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:13.553106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e17e-6275-4b9d-a2da-87652e070a09 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.553400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e17e-6275-4b9d-a2da-87652e070a09 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.553707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e17e-6275-4b9d-a2da-87652e070a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.583889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-31a6e17e-6275-4b9d-a2da-87652e070a09 None None] MFA Rules not processed for user `da93146b1fc14bf4a9595bb6576fbf98`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:13.585103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-31a6e17e-6275-4b9d-a2da-87652e070a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:13.650323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1191/2410] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:13.660235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d12b62-c990-4101-9734-c23a2d61101a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.727443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d12b62-c990-4101-9734-c23a2d61101a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.733140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce8ae4-60c1-45fb-aa31-5407f07b2277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.733848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8ae4-60c1-45fb-aa31-5407f07b2277 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.734197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8ae4-60c1-45fb-aa31-5407f07b2277 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.734536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce8ae4-60c1-45fb-aa31-5407f07b2277 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.735373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce8ae4-60c1-45fb-aa31-5407f07b2277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.737347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ce8ae4-60c1-45fb-aa31-5407f07b2277 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.769655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1220/2411] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:13 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:13.770807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e17e-6275-4b9d-a2da-87652e070a09 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.782509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e17e-6275-4b9d-a2da-87652e070a09 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.841191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f01b4-cff6-4941-8d04-7f9ddf2f8571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.841462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f01b4-cff6-4941-8d04-7f9ddf2f8571 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.842101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f01b4-cff6-4941-8d04-7f9ddf2f8571 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.842101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f01b4-cff6-4941-8d04-7f9ddf2f8571 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.842432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f01b4-cff6-4941-8d04-7f9ddf2f8571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.844347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f01b4-cff6-4941-8d04-7f9ddf2f8571 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.876042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1192/2412] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:13 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:13.877513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce8ae4-60c1-45fb-aa31-5407f07b2277 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.888571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ce8ae4-60c1-45fb-aa31-5407f07b2277 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.893514 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64639b70-f8c6-4524-8437-2802e91d9ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.894184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64639b70-f8c6-4524-8437-2802e91d9ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.894792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64639b70-f8c6-4524-8437-2802e91d9ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.894792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64639b70-f8c6-4524-8437-2802e91d9ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.895507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64639b70-f8c6-4524-8437-2802e91d9ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.897222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64639b70-f8c6-4524-8437-2802e91d9ba3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.908564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1221/2413] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:13.912812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f01b4-cff6-4941-8d04-7f9ddf2f8571 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.923698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f01b4-cff6-4941-8d04-7f9ddf2f8571 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:13.928348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7589ee0-92ee-48b2-809d-7b3f57ddbc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:13.928994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7589ee0-92ee-48b2-809d-7b3f57ddbc92 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:13.929273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7589ee0-92ee-48b2-809d-7b3f57ddbc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:13.929547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7589ee0-92ee-48b2-809d-7b3f57ddbc92 admin admin] PATH_INFO: `/v3/projects/41ad75f39de244d099e4897eb3624e2c/users/be43f22232a44097b9bc374da2c69e58/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:13.945267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7589ee0-92ee-48b2-809d-7b3f57ddbc92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ad75f39de244d099e4897eb3624e2c, user_id=be43f22232a44097b9bc374da2c69e58, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:13.947991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7589ee0-92ee-48b2-809d-7b3f57ddbc92 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:13.961583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1193/2414] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:13 2022] PUT /identity/v3/projects/41ad75f39de244d099e4897eb3624e2c/users/be43f22232a44097b9bc374da2c69e58/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:13.965694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64639b70-f8c6-4524-8437-2802e91d9ba3 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:13.970799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:13.977633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64639b70-f8c6-4524-8437-2802e91d9ba3 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:14.037699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a958881c-7fd0-4daf-869a-170916b8ed4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:14.038284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958881c-7fd0-4daf-869a-170916b8ed4c admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:14.038566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958881c-7fd0-4daf-869a-170916b8ed4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:14.038841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a958881c-7fd0-4daf-869a-170916b8ed4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:14.039485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958881c-7fd0-4daf-869a-170916b8ed4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:14.041084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a958881c-7fd0-4daf-869a-170916b8ed4c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:14.051143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1222/2415] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:14.055281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7589ee0-92ee-48b2-809d-7b3f57ddbc92 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:14.066369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7589ee0-92ee-48b2-809d-7b3f57ddbc92 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:14.070981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:14.071610 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:14.071888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:14.072159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] PATH_INFO: `/v3/projects/41ad75f39de244d099e4897eb3624e2c/users/be43f22232a44097b9bc374da2c69e58/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:14.074059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ad75f39de244d099e4897eb3624e2c, user_id=be43f22232a44097b9bc374da2c69e58, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:14.076750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:14.086641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1194/2416] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:14 2022] PUT /identity/v3/projects/41ad75f39de244d099e4897eb3624e2c/users/be43f22232a44097b9bc374da2c69e58/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:14.097978 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028fa99-bbaa-451b-9283-fd3b66cb3be3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:14.098267 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028fa99-bbaa-451b-9283-fd3b66cb3be3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:14.098541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028fa99-bbaa-451b-9283-fd3b66cb3be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:14.127670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-6028fa99-bbaa-451b-9283-fd3b66cb3be3 None None] MFA Rules not processed for user `be43f22232a44097b9bc374da2c69e58`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:14.128912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6028fa99-bbaa-451b-9283-fd3b66cb3be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:14.195619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1223/2417] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:14.206212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:14.207889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:14.214940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:14.266769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23be42f-61e0-452c-8044-793182b2c016 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:14.270557 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:14.271153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:14.271430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:14.271700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:14.281707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:14.283135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:14.292962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1195/2418] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:14.393825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028fa99-bbaa-451b-9283-fd3b66cb3be3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:14.403685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028fa99-bbaa-451b-9283-fd3b66cb3be3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:14.407372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:14.408039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:14.408320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:14.408590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:14.464022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:14.466093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:14.475841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1224/2419] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:15.052867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:15.054436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:15.154952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5d56a-c9cb-479c-b9cb-3246b9b366ea service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:15.160540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:15.161150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:15.161432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:15.161744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:15.163505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:15.175367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:15.176954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:15.187832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1196/2420] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:19.171914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:19.173248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:19.203290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:19.245879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746420e8-69fb-4a17-ad42-3c27e7cabb2f service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:19.252304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:19.253164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:19.253402 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:19.253569 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] PATH_INFO: `/v3/users/17d37e2bf6324ee88f9f70da52cc63f4` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:19.258444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d37e2bf6324ee88f9f70da52cc63f4)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:19.260582 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:19.264941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:19.279662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142606e-5d6c-444a-9bf6-1fa1b97532c4 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:19.284253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:19.284878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:19.285161 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:19.285433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] PATH_INFO: `/v3/users/da93146b1fc14bf4a9595bb6576fbf98` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:19.286613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da93146b1fc14bf4a9595bb6576fbf98)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:19.288848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:19.313843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d37e2bf6324ee88f9f70da52cc63f4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.321716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d37e2bf6324ee88f9f70da52cc63f4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.322398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d37e2bf6324ee88f9f70da52cc63f4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.335476 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d37e2bf6324ee88f9f70da52cc63f4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.338632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da93146b1fc14bf4a9595bb6576fbf98'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.343966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6e0d9d76-ca53-4848-889d-05babd311605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d37e2bf6324ee88f9f70da52cc63f4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.346226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da93146b1fc14bf4a9595bb6576fbf98'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.346934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da93146b1fc14bf4a9595bb6576fbf98'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.350354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1225/2421] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:19 2022] DELETE /identity/v3/users/17d37e2bf6324ee88f9f70da52cc63f4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:19.354541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443966-1e51-49d3-9ae4-fabee801e0d0 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:19.354825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443966-1e51-49d3-9ae4-fabee801e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:19.355083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72443966-1e51-49d3-9ae4-fabee801e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:19.359967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da93146b1fc14bf4a9595bb6576fbf98'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.367288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c4198362-d769-4c16-a44c-c22707ccc464 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da93146b1fc14bf4a9595bb6576fbf98'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:19.374377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-72443966-1e51-49d3-9ae4-fabee801e0d0 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:19.374539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1197/2422] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:19 2022] DELETE /identity/v3/users/da93146b1fc14bf4a9595bb6576fbf98 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:19.378445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-72443966-1e51-49d3-9ae4-fabee801e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:19.378607 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2715063-2c16-4fbe-9390-f5682928a39d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:19.378843 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2715063-2c16-4fbe-9390-f5682928a39d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:19.379124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2715063-2c16-4fbe-9390-f5682928a39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:19.388119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:19.400895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-c2715063-2c16-4fbe-9390-f5682928a39d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:19.402389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c2715063-2c16-4fbe-9390-f5682928a39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:19.439441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1226/2423] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:19.449977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72443966-1e51-49d3-9ae4-fabee801e0d0 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:19.451362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-72443966-1e51-49d3-9ae4-fabee801e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:19.464560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1198/2424] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:19.474068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2715063-2c16-4fbe-9390-f5682928a39d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:19.475393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c2715063-2c16-4fbe-9390-f5682928a39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:19.497536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:19.515346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72443966-1e51-49d3-9ae4-fabee801e0d0 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:19.519414 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:19.520062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:19.520341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:19.520615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:19.531882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:19.533526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:19.539890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2715063-2c16-4fbe-9390-f5682928a39d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:19.543553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1227/2425] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:19.543708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:19.544258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:19.544540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:19.544814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:19.554338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:19.555864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:19.567008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1199/2426] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:20.048757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.050491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:20.065819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde016fe-f3a7-414b-8bd2-c5fc8fab32b1 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.078222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.079725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:20.080014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.080766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.081244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.081513 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] PATH_INFO: `/v3/projects/c8f02786494b4a1a901c0d6240921180` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.083325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f02786494b4a1a901c0d6240921180)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.086651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.110440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b658f8d-ec2a-4dd1-9e29-65abd56d1a15 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.123674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.125759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.126050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.126329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] PATH_INFO: `/v3/projects/894e75d31e644dbbbfbfe9730ccaecd0` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.128174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=894e75d31e644dbbbfbfe9730ccaecd0)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.130194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.183772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f02786494b4a1a901c0d6240921180', 'request_id': 'req-8cfcc891-36ce-4cc2-866b-b14e9f70b470'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.190191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '894e75d31e644dbbbfbfe9730ccaecd0', 'request_id': 'req-c8d884ce-327e-48c7-9ebf-463581e6f3a6'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.196245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f02786494b4a1a901c0d6240921180', 'request_id': 'req-8cfcc891-36ce-4cc2-866b-b14e9f70b470'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.196848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '894e75d31e644dbbbfbfe9730ccaecd0', 'request_id': 'req-c8d884ce-327e-48c7-9ebf-463581e6f3a6'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.203771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '894e75d31e644dbbbfbfe9730ccaecd0', 'request_id': 'req-c8d884ce-327e-48c7-9ebf-463581e6f3a6'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.208041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f02786494b4a1a901c0d6240921180', 'request_id': 'req-8cfcc891-36ce-4cc2-866b-b14e9f70b470'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.233192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] The token cache is being invalidate because project 894e75d31e644dbbbfbfe9730ccaecd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:20.233468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.233849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] The token cache is being invalidate because project c8f02786494b4a1a901c0d6240921180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:20.234090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.234208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.234730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8cfcc891-36ce-4cc2-866b-b14e9f70b470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.239045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1200/2427] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:20 2022] DELETE /identity/v3/projects/894e75d31e644dbbbfbfe9730ccaecd0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:20.239264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1228/2428] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:20 2022] DELETE /identity/v3/projects/c8f02786494b4a1a901c0d6240921180 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:20.239882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.241082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b016e-49ad-4dd5-80a4-09a7025ad739 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.241217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:20.241389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b016e-49ad-4dd5-80a4-09a7025ad739 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.241675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b016e-49ad-4dd5-80a4-09a7025ad739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.266120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-8c8b016e-49ad-4dd5-80a4-09a7025ad739 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:20.267337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8c8b016e-49ad-4dd5-80a4-09a7025ad739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:20.286060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:20.338828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d884ce-327e-48c7-9ebf-463581e6f3a6 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.341758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1229/2429] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:20.348170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd196336-f0f1-463f-afb0-7ecac4f30c51 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.348551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd196336-f0f1-463f-afb0-7ecac4f30c51 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.348905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd196336-f0f1-463f-afb0-7ecac4f30c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.353934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.354614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.354897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.355175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] PATH_INFO: `/v3/users/be43f22232a44097b9bc374da2c69e58` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.356641 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be43f22232a44097b9bc374da2c69e58)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.358774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.373324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fd196336-f0f1-463f-afb0-7ecac4f30c51 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:20.374742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fd196336-f0f1-463f-afb0-7ecac4f30c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:20.388797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1230/2430] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:20.390375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd196336-f0f1-463f-afb0-7ecac4f30c51 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.402230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd196336-f0f1-463f-afb0-7ecac4f30c51 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.407681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.408296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.408572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.408853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.409482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:20.409833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.411686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.428928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1231/2431] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:25:20.430852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.443757 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be43f22232a44097b9bc374da2c69e58'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.452776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108075c3-c0d4-40a4-97bc-20a786a6cc92 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.455002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be43f22232a44097b9bc374da2c69e58'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.455999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be43f22232a44097b9bc374da2c69e58'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.461900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecf0dd-6ded-45ad-b29a-ac6cc962ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.462705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecf0dd-6ded-45ad-b29a-ac6cc962ed30 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.462994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecf0dd-6ded-45ad-b29a-ac6cc962ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.463274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecf0dd-6ded-45ad-b29a-ac6cc962ed30 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.463966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecf0dd-6ded-45ad-b29a-ac6cc962ed30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.466182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecf0dd-6ded-45ad-b29a-ac6cc962ed30 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.473350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be43f22232a44097b9bc374da2c69e58'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.483091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be43f22232a44097b9bc374da2c69e58'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.491398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1201/2432] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:20 2022] DELETE /identity/v3/users/be43f22232a44097b9bc374da2c69e58 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:20.512325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1232/2433] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:20 2022] POST /identity/v3/projects => generated 367 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:20.517082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.518616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:20.597443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05327bc1-a11d-459d-821d-38822d751757 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.669938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:20.679276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514c62a-0a24-414f-a454-3ce76a1b318b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.680738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c62a-0a24-414f-a454-3ce76a1b318b admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.681105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c62a-0a24-414f-a454-3ce76a1b318b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.681498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a514c62a-0a24-414f-a454-3ce76a1b318b admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.682249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514c62a-0a24-414f-a454-3ce76a1b318b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.685680 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a514c62a-0a24-414f-a454-3ce76a1b318b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.722378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1202/2434] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:20 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:20.723708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecf0dd-6ded-45ad-b29a-ac6cc962ed30 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.736836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecf0dd-6ded-45ad-b29a-ac6cc962ed30 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.742215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.742822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.743104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.743373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.744016 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.745974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.759525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1233/2435] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:20.761665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514c62a-0a24-414f-a454-3ce76a1b318b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.773478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a514c62a-0a24-414f-a454-3ce76a1b318b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.779315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de953-3b17-45d9-8028-f22cf26839bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.779923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de953-3b17-45d9-8028-f22cf26839bd admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.780199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de953-3b17-45d9-8028-f22cf26839bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.780470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de953-3b17-45d9-8028-f22cf26839bd admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.781141 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3de953-3b17-45d9-8028-f22cf26839bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.782854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3de953-3b17-45d9-8028-f22cf26839bd admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.796935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1203/2436] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:20.798773 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.798917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de953-3b17-45d9-8028-f22cf26839bd admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:20.804253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:20.823873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de953-3b17-45d9-8028-f22cf26839bd admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.825005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88e3d5-5694-4660-befd-d1c7e456b8eb admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:20.830929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.832557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.833257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.834024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda6da9-1d46-43e4-ae16-cf5adc445e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:20.834684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda6da9-1d46-43e4-ae16-cf5adc445e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.834986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda6da9-1d46-43e4-ae16-cf5adc445e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.835260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda6da9-1d46-43e4-ae16-cf5adc445e0d admin admin] PATH_INFO: `/v3/projects/f5dd48d04bc44304b41c9c51a1f1bac5/users/c535b1b4c45b47c3a97fa698bbc772bc/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.836253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] PATH_INFO: `/v3/projects/41ad75f39de244d099e4897eb3624e2c` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.837405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ad75f39de244d099e4897eb3624e2c)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.839990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.851527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:20.855284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda6da9-1d46-43e4-ae16-cf5adc445e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5dd48d04bc44304b41c9c51a1f1bac5, user_id=c535b1b4c45b47c3a97fa698bbc772bc, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:20.858704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eda6da9-1d46-43e4-ae16-cf5adc445e0d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:20.880171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1204/2437] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:20 2022] PUT /identity/v3/projects/f5dd48d04bc44304b41c9c51a1f1bac5/users/c535b1b4c45b47c3a97fa698bbc772bc/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:20.891083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab824a-a3a1-46ce-b70c-67f5e9d6b6fb None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.891433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab824a-a3a1-46ce-b70c-67f5e9d6b6fb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.891770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eab824a-a3a1-46ce-b70c-67f5e9d6b6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.893712 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ad75f39de244d099e4897eb3624e2c', 'request_id': 'req-21e9f714-d9cd-4d23-8960-0898c3bbf30e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.900360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ad75f39de244d099e4897eb3624e2c', 'request_id': 'req-21e9f714-d9cd-4d23-8960-0898c3bbf30e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.908035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ad75f39de244d099e4897eb3624e2c', 'request_id': 'req-21e9f714-d9cd-4d23-8960-0898c3bbf30e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.927973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2eab824a-a3a1-46ce-b70c-67f5e9d6b6fb None None] MFA Rules not processed for user `c535b1b4c45b47c3a97fa698bbc772bc`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:20.931334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2eab824a-a3a1-46ce-b70c-67f5e9d6b6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:20.932205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] The token cache is being invalidate because project 41ad75f39de244d099e4897eb3624e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:20.933499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.934590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-21e9f714-d9cd-4d23-8960-0898c3bbf30e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:20.937362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1234/2438] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:20 2022] DELETE /identity/v3/projects/41ad75f39de244d099e4897eb3624e2c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:20.944741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519220-ecfe-4236-88b8-789eae0c79b2 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:20.946008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519220-ecfe-4236-88b8-789eae0c79b2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:20.946290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519220-ecfe-4236-88b8-789eae0c79b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:20.978226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3f519220-ecfe-4236-88b8-789eae0c79b2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:20.982810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3f519220-ecfe-4236-88b8-789eae0c79b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:21.028012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1205/2439] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:21.042152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab824a-a3a1-46ce-b70c-67f5e9d6b6fb None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.043835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2eab824a-a3a1-46ce-b70c-67f5e9d6b6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:21.074939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1235/2440] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:21.083161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d89a5-fc55-4acd-be42-81bed5f6e9b3 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.083686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d89a5-fc55-4acd-be42-81bed5f6e9b3 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.084022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d89a5-fc55-4acd-be42-81bed5f6e9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.107714 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-737d89a5-fc55-4acd-be42-81bed5f6e9b3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:21.109275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-737d89a5-fc55-4acd-be42-81bed5f6e9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:21.113871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eab824a-a3a1-46ce-b70c-67f5e9d6b6fb None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.117882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d4a8c-5f45-431f-ab20-a4a1467f8229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.118905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d4a8c-5f45-431f-ab20-a4a1467f8229 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.119047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1236/2441] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:21.119144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d4a8c-5f45-431f-ab20-a4a1467f8229 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.119144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d4a8c-5f45-431f-ab20-a4a1467f8229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.124090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d89a5-fc55-4acd-be42-81bed5f6e9b3 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.130775 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d4a8c-5f45-431f-ab20-a4a1467f8229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.132429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5d4a8c-5f45-431f-ab20-a4a1467f8229 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.133440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d89a5-fc55-4acd-be42-81bed5f6e9b3 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.140592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.141207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.141485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.141855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.142437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:21.142729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.144426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.154023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1206/2442] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:21.160357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1237/2443] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:25:21.161460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d4a8c-5f45-431f-ab20-a4a1467f8229 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.173762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d4a8c-5f45-431f-ab20-a4a1467f8229 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.179793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dcd8b-3795-48cd-8e3b-05d6e95357f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.180621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dcd8b-3795-48cd-8e3b-05d6e95357f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.181590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dcd8b-3795-48cd-8e3b-05d6e95357f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.182097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989dcd8b-3795-48cd-8e3b-05d6e95357f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.183460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dcd8b-3795-48cd-8e3b-05d6e95357f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.185764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989dcd8b-3795-48cd-8e3b-05d6e95357f8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.205643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.207795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:21.229108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1207/2444] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:21 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:21.234135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dcd8b-3795-48cd-8e3b-05d6e95357f8 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.266997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:21.312592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb903e41-82a4-4d08-a797-7c3886e03984 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.313300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989dcd8b-3795-48cd-8e3b-05d6e95357f8 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.319071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ad37-2081-4a3e-b552-26e10370d5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.319699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ad37-2081-4a3e-b552-26e10370d5be admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.320407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ad37-2081-4a3e-b552-26e10370d5be admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.320407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ad37-2081-4a3e-b552-26e10370d5be admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.320867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928ad37-2081-4a3e-b552-26e10370d5be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.322982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928ad37-2081-4a3e-b552-26e10370d5be admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.331063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:21.352879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1208/2445] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:21 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:21.353960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ad37-2081-4a3e-b552-26e10370d5be admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.372048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ad37-2081-4a3e-b552-26e10370d5be admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.377962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab879-bba3-4425-87a4-6d1d60adc266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.378694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab879-bba3-4425-87a4-6d1d60adc266 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.379084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab879-bba3-4425-87a4-6d1d60adc266 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.379665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab879-bba3-4425-87a4-6d1d60adc266 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.380453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ab879-bba3-4425-87a4-6d1d60adc266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.382767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ab879-bba3-4425-87a4-6d1d60adc266 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.401743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1209/2446] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:21.410891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab879-bba3-4425-87a4-6d1d60adc266 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.424543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab879-bba3-4425-87a4-6d1d60adc266 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.434396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1857027-363d-4982-abe8-f1aa19141f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.435131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1857027-363d-4982-abe8-f1aa19141f48 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.435488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1857027-363d-4982-abe8-f1aa19141f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.435832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1857027-363d-4982-abe8-f1aa19141f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.436892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1857027-363d-4982-abe8-f1aa19141f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.437928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736cf5e2-8254-4fdb-a71d-1d5c5e355f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.438563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cf5e2-8254-4fdb-a71d-1d5c5e355f74 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.438926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cf5e2-8254-4fdb-a71d-1d5c5e355f74 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.439122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1857027-363d-4982-abe8-f1aa19141f48 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.439255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736cf5e2-8254-4fdb-a71d-1d5c5e355f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.441360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-736cf5e2-8254-4fdb-a71d-1d5c5e355f74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:21.469272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1210/2447] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:21.469272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1857027-363d-4982-abe8-f1aa19141f48 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.476353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cf5e2-8254-4fdb-a71d-1d5c5e355f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.483488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1857027-363d-4982-abe8-f1aa19141f48 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.498930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74d458-0216-46bf-9d19-36dd1300ed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.502070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736cf5e2-8254-4fdb-a71d-1d5c5e355f74 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.513274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d458-0216-46bf-9d19-36dd1300ed56 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.513577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d458-0216-46bf-9d19-36dd1300ed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.516849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74d458-0216-46bf-9d19-36dd1300ed56 admin admin] PATH_INFO: `/v3/projects/1d2b6558bba24b1cb69e601455d85579/users/fe34ba5ee4024c5cb5f5907be4f83135/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.535143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1238/2448] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:21.554029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74d458-0216-46bf-9d19-36dd1300ed56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d2b6558bba24b1cb69e601455d85579, user_id=fe34ba5ee4024c5cb5f5907be4f83135, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.556783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c74d458-0216-46bf-9d19-36dd1300ed56 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.575829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1211/2449] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:21 2022] PUT /identity/v3/projects/1d2b6558bba24b1cb69e601455d85579/users/fe34ba5ee4024c5cb5f5907be4f83135/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:21.586949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453381f-b2da-448f-836c-ccbf17da8a93 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.587553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453381f-b2da-448f-836c-ccbf17da8a93 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.587997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453381f-b2da-448f-836c-ccbf17da8a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.642376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-1453381f-b2da-448f-836c-ccbf17da8a93 None None] MFA Rules not processed for user `fe34ba5ee4024c5cb5f5907be4f83135`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:21.643670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1453381f-b2da-448f-836c-ccbf17da8a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:21.735387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1239/2450] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:21.740036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74d458-0216-46bf-9d19-36dd1300ed56 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.809018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74d458-0216-46bf-9d19-36dd1300ed56 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.813842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cef77-f0c1-4ddc-bd5f-c1073259eff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.814469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cef77-f0c1-4ddc-bd5f-c1073259eff7 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.814703 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cef77-f0c1-4ddc-bd5f-c1073259eff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.814984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cef77-f0c1-4ddc-bd5f-c1073259eff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.815665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cef77-f0c1-4ddc-bd5f-c1073259eff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.817692 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cef77-f0c1-4ddc-bd5f-c1073259eff7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.848911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1212/2451] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:21 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:21.852887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453381f-b2da-448f-836c-ccbf17da8a93 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.865971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453381f-b2da-448f-836c-ccbf17da8a93 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.930607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94455a-382c-40da-9aa2-8d1847cabeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.931165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94455a-382c-40da-9aa2-8d1847cabeea admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.931522 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94455a-382c-40da-9aa2-8d1847cabeea admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.931864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94455a-382c-40da-9aa2-8d1847cabeea admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.932602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94455a-382c-40da-9aa2-8d1847cabeea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.934625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94455a-382c-40da-9aa2-8d1847cabeea admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.959881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1240/2452] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:21 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:21.965062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cef77-f0c1-4ddc-bd5f-c1073259eff7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:21.976285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cef77-f0c1-4ddc-bd5f-c1073259eff7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:21.981231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429f8a3-960b-4aff-a2c9-1a6840c019c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:21.981883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429f8a3-960b-4aff-a2c9-1a6840c019c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:21.982168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429f8a3-960b-4aff-a2c9-1a6840c019c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:21.982450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429f8a3-960b-4aff-a2c9-1a6840c019c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:21.983084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429f8a3-960b-4aff-a2c9-1a6840c019c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:21.984717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429f8a3-960b-4aff-a2c9-1a6840c019c8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:21.996325 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1213/2453] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:22.000482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94455a-382c-40da-9aa2-8d1847cabeea admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:22.015271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94455a-382c-40da-9aa2-8d1847cabeea admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:22.024943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc31b0-755c-4a24-9935-63441a8ff39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:22.025806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc31b0-755c-4a24-9935-63441a8ff39c admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:22.026153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc31b0-755c-4a24-9935-63441a8ff39c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:22.027076 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc31b0-755c-4a24-9935-63441a8ff39c admin admin] PATH_INFO: `/v3/projects/bf1cd4714abf4b72a57aba20df6666f7/users/5030f88b6a454677aad3c1e71162e98f/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:22.044875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc31b0-755c-4a24-9935-63441a8ff39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1cd4714abf4b72a57aba20df6666f7, user_id=5030f88b6a454677aad3c1e71162e98f, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:22.048060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc31b0-755c-4a24-9935-63441a8ff39c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:22.059814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:22.067705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1241/2454] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:21 2022] PUT /identity/v3/projects/bf1cd4714abf4b72a57aba20df6666f7/users/5030f88b6a454677aad3c1e71162e98f/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:22.071518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429f8a3-960b-4aff-a2c9-1a6840c019c8 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:22.083300 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429f8a3-960b-4aff-a2c9-1a6840c019c8 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:22.128381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:22.150698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb3057-9bd6-452e-a428-dfeb1323c9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:22.151980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3057-9bd6-452e-a428-dfeb1323c9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:22.152261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3057-9bd6-452e-a428-dfeb1323c9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:22.152531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb3057-9bd6-452e-a428-dfeb1323c9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:22.153169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb3057-9bd6-452e-a428-dfeb1323c9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:22.154779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb3057-9bd6-452e-a428-dfeb1323c9e7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:22.164884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1214/2455] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:22.169186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc31b0-755c-4a24-9935-63441a8ff39c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:22.179746 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc31b0-755c-4a24-9935-63441a8ff39c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:22.185114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:22.185812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:22.186093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:22.186373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] PATH_INFO: `/v3/projects/bf1cd4714abf4b72a57aba20df6666f7/users/5030f88b6a454677aad3c1e71162e98f/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:22.188341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1cd4714abf4b72a57aba20df6666f7, user_id=5030f88b6a454677aad3c1e71162e98f, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:22.191535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:22.203820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1242/2456] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:22 2022] PUT /identity/v3/projects/bf1cd4714abf4b72a57aba20df6666f7/users/5030f88b6a454677aad3c1e71162e98f/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:22.215993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ce9d5-4757-45ae-9e2e-6f58042c3f93 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:22.216293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ce9d5-4757-45ae-9e2e-6f58042c3f93 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:22.216578 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ce9d5-4757-45ae-9e2e-6f58042c3f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:22.248898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-805ce9d5-4757-45ae-9e2e-6f58042c3f93 None None] MFA Rules not processed for user `5030f88b6a454677aad3c1e71162e98f`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:22.250168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-805ce9d5-4757-45ae-9e2e-6f58042c3f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:22.321947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1215/2457] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:22.333409 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:22.335144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:22.397250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582e712-c15d-4af1-8ae2-b309af8089d9 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:22.401464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d19ea6-7633-4d6d-aee8-5cf2683ac77b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:22.402486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d19ea6-7633-4d6d-aee8-5cf2683ac77b service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:22.402771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d19ea6-7633-4d6d-aee8-5cf2683ac77b service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:22.403048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d19ea6-7633-4d6d-aee8-5cf2683ac77b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:22.413295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d19ea6-7633-4d6d-aee8-5cf2683ac77b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:22.414781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d19ea6-7633-4d6d-aee8-5cf2683ac77b service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:22.437005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1243/2458] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:23.128240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ce9d5-4757-45ae-9e2e-6f58042c3f93 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:23.190279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ce9d5-4757-45ae-9e2e-6f58042c3f93 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:23.194810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:23.195459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:23.195728 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:23.196005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] PATH_INFO: `/v3/users/fe34ba5ee4024c5cb5f5907be4f83135` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:23.197254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe34ba5ee4024c5cb5f5907be4f83135)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:23.199294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:23.246485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe34ba5ee4024c5cb5f5907be4f83135'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.254376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe34ba5ee4024c5cb5f5907be4f83135'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.255079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe34ba5ee4024c5cb5f5907be4f83135'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.262698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:23.268568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe34ba5ee4024c5cb5f5907be4f83135'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.277163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe34ba5ee4024c5cb5f5907be4f83135'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.283048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1216/2459] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:23 2022] DELETE /identity/v3/users/fe34ba5ee4024c5cb5f5907be4f83135 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:23.287548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb309-5cae-4fc6-b985-e814f14ad726 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:23.287842 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb309-5cae-4fc6-b985-e814f14ad726 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:23.288120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb309-5cae-4fc6-b985-e814f14ad726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:23.307417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-50cfb309-5cae-4fc6-b985-e814f14ad726 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:23.308638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-50cfb309-5cae-4fc6-b985-e814f14ad726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:23.367183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1244/2460] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:23.377419 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:23.378884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:23.446217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82d8428-d7ce-4476-b031-51907d486f30 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:23.450421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352cc8-256c-4b62-9ffd-b749cf3af95d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:23.451023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352cc8-256c-4b62-9ffd-b749cf3af95d service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:23.451303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352cc8-256c-4b62-9ffd-b749cf3af95d service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:23.451576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b352cc8-256c-4b62-9ffd-b749cf3af95d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:23.460392 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b352cc8-256c-4b62-9ffd-b749cf3af95d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:23.461852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b352cc8-256c-4b62-9ffd-b749cf3af95d service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:23.471796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1217/2461] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:23.738995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb309-5cae-4fc6-b985-e814f14ad726 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:23.740552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-50cfb309-5cae-4fc6-b985-e814f14ad726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:23.753023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb309-5cae-4fc6-b985-e814f14ad726 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:23.757690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:23.758318 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:23.758595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:23.758865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] PATH_INFO: `/v3/projects/1d2b6558bba24b1cb69e601455d85579` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:23.760036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d2b6558bba24b1cb69e601455d85579)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:23.762514 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:23.809357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d2b6558bba24b1cb69e601455d85579', 'request_id': 'req-00de2043-49ac-4a7e-913e-f59496b7b6fd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.814912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d2b6558bba24b1cb69e601455d85579', 'request_id': 'req-00de2043-49ac-4a7e-913e-f59496b7b6fd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.821412 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d2b6558bba24b1cb69e601455d85579', 'request_id': 'req-00de2043-49ac-4a7e-913e-f59496b7b6fd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.834227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] The token cache is being invalidate because project 1d2b6558bba24b1cb69e601455d85579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:23.834512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.835124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.836628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1245/2462] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:23 2022] DELETE /identity/v3/projects/1d2b6558bba24b1cb69e601455d85579 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:23.841243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352cc8-256c-4b62-9ffd-b749cf3af95d service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:23.850624 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b352cc8-256c-4b62-9ffd-b749cf3af95d service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:23.906170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:23.906718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:23.906996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:23.907267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] PATH_INFO: `/v3/users/5030f88b6a454677aad3c1e71162e98f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:23.908374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5030f88b6a454677aad3c1e71162e98f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:23.910344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:23.957354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5030f88b6a454677aad3c1e71162e98f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.965126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5030f88b6a454677aad3c1e71162e98f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.966337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5030f88b6a454677aad3c1e71162e98f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.984531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5030f88b6a454677aad3c1e71162e98f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:23.989330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:23.996625 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a3bdbc81-4fdd-4809-ada7-2e578a23c1b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5030f88b6a454677aad3c1e71162e98f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:24.012985 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1218/2463] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:23 2022] DELETE /identity/v3/users/5030f88b6a454677aad3c1e71162e98f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:24.300210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:24.301948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:24.317236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de2043-49ac-4a7e-913e-f59496b7b6fd admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:24.329973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:24.395251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:24.396084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.396519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.396893 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] PATH_INFO: `/v3/projects/bf1cd4714abf4b72a57aba20df6666f7` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.398306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1cd4714abf4b72a57aba20df6666f7)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:24.400549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:24.452660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1cd4714abf4b72a57aba20df6666f7', 'request_id': 'req-0b871d06-8f16-49b2-a9ad-856b85e6d204'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:24.458876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1cd4714abf4b72a57aba20df6666f7', 'request_id': 'req-0b871d06-8f16-49b2-a9ad-856b85e6d204'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:24.466066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1cd4714abf4b72a57aba20df6666f7', 'request_id': 'req-0b871d06-8f16-49b2-a9ad-856b85e6d204'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:24.479480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] The token cache is being invalidate because project bf1cd4714abf4b72a57aba20df6666f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:24.480520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:24.481251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0b871d06-8f16-49b2-a9ad-856b85e6d204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:24.482760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1246/2464] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:24 2022] DELETE /identity/v3/projects/bf1cd4714abf4b72a57aba20df6666f7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:24.487803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2cca-12dd-4f0a-abca-5a034dadf094 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.488304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2cca-12dd-4f0a-abca-5a034dadf094 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.488517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d2cca-12dd-4f0a-abca-5a034dadf094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.510204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-680d2cca-12dd-4f0a-abca-5a034dadf094 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:24.511929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-680d2cca-12dd-4f0a-abca-5a034dadf094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:24.570463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1219/2465] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:24.579111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa08c0-42f9-4828-ad06-caee9a59af4c None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.579470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa08c0-42f9-4828-ad06-caee9a59af4c None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.579824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa08c0-42f9-4828-ad06-caee9a59af4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.600946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-6cfa08c0-42f9-4828-ad06-caee9a59af4c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:24.602191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6cfa08c0-42f9-4828-ad06-caee9a59af4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:24.610309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1247/2466] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:24.614613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d2cca-12dd-4f0a-abca-5a034dadf094 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:24.625304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d2cca-12dd-4f0a-abca-5a034dadf094 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:24.630150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:24.630753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.631034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.631307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.631982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:24.632287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:24.634008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:24.645933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1220/2467] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:25:24.650129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa08c0-42f9-4828-ad06-caee9a59af4c None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:24.659477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa08c0-42f9-4828-ad06-caee9a59af4c None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:24.664462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0a54b-e0c8-4dae-a857-37e77b73f85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:24.664999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0a54b-e0c8-4dae-a857-37e77b73f85a admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.665284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0a54b-e0c8-4dae-a857-37e77b73f85a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.665555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0a54b-e0c8-4dae-a857-37e77b73f85a admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.666218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0a54b-e0c8-4dae-a857-37e77b73f85a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:24.668178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0a54b-e0c8-4dae-a857-37e77b73f85a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:24.696295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1248/2468] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:24 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:24.701360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:24.711519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b8281-5d8d-481b-aeda-c2a9f891c02a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:24.769700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb33a6c-14eb-43b9-abac-53f405fe6ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:24.770298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb33a6c-14eb-43b9-abac-53f405fe6ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.770577 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb33a6c-14eb-43b9-abac-53f405fe6ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.770849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb33a6c-14eb-43b9-abac-53f405fe6ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.771482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb33a6c-14eb-43b9-abac-53f405fe6ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:24.773412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb33a6c-14eb-43b9-abac-53f405fe6ae7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:24.794091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1221/2469] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:24 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:24.798135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0a54b-e0c8-4dae-a857-37e77b73f85a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:24.808635 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0a54b-e0c8-4dae-a857-37e77b73f85a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:24.813386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee12ac-2648-49c9-b32b-921b23c39779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:24.814040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee12ac-2648-49c9-b32b-921b23c39779 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.814319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee12ac-2648-49c9-b32b-921b23c39779 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.814590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fee12ac-2648-49c9-b32b-921b23c39779 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.815259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee12ac-2648-49c9-b32b-921b23c39779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:24.816867 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fee12ac-2648-49c9-b32b-921b23c39779 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:24.827804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1249/2470] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:24.831556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb33a6c-14eb-43b9-abac-53f405fe6ae7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:24.841866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb33a6c-14eb-43b9-abac-53f405fe6ae7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:24.846511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c47eb-324a-4b27-97dc-08635fa322ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:24.847098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c47eb-324a-4b27-97dc-08635fa322ea admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.847381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c47eb-324a-4b27-97dc-08635fa322ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.847654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c47eb-324a-4b27-97dc-08635fa322ea admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.848289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c47eb-324a-4b27-97dc-08635fa322ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:24.849977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c47eb-324a-4b27-97dc-08635fa322ea admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:24.860412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1222/2471] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:24.864275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee12ac-2648-49c9-b32b-921b23c39779 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:24.874948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fee12ac-2648-49c9-b32b-921b23c39779 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:24.879690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:24.880341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.880621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.880895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] PATH_INFO: `/v3/projects/017865427104453898d35dcd902cf497/users/0b48e5e48d2b442ba3dd6ce0dab533ee/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.895729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017865427104453898d35dcd902cf497, user_id=0b48e5e48d2b442ba3dd6ce0dab533ee, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:24.898445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:24.910741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1250/2472] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:24 2022] PUT /identity/v3/projects/017865427104453898d35dcd902cf497/users/0b48e5e48d2b442ba3dd6ce0dab533ee/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:24.915864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f613b7c-abcd-4708-87b8-8565c1226671 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:24.916093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f613b7c-abcd-4708-87b8-8565c1226671 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:24.916370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f613b7c-abcd-4708-87b8-8565c1226671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:24.935850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8f613b7c-abcd-4708-87b8-8565c1226671 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:24.937055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8f613b7c-abcd-4708-87b8-8565c1226671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:24.986329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:24.996934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1223/2473] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:25.006395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:25.008093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:25.049474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:25.066036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1b0ce-e5e0-4dea-8b2a-c9ca783245f8 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:25.069697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a58658-7c63-4551-94a2-8a8b5cf72446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:25.070302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a58658-7c63-4551-94a2-8a8b5cf72446 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:25.070576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a58658-7c63-4551-94a2-8a8b5cf72446 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:25.070846 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a58658-7c63-4551-94a2-8a8b5cf72446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:25.079616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a58658-7c63-4551-94a2-8a8b5cf72446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:25.081037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a58658-7c63-4551-94a2-8a8b5cf72446 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:25.090523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1251/2474] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:25.746175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57c734-05ce-4d9d-8de2-c4a5d5bde581 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:25.746707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57c734-05ce-4d9d-8de2-c4a5d5bde581 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:25.746925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57c734-05ce-4d9d-8de2-c4a5d5bde581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:25.786658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-5e57c734-05ce-4d9d-8de2-c4a5d5bde581 None None] MFA Rules not processed for user `0b48e5e48d2b442ba3dd6ce0dab533ee`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:25.788036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5e57c734-05ce-4d9d-8de2-c4a5d5bde581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:25.865330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1224/2475] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:25.870108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a58658-7c63-4551-94a2-8a8b5cf72446 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:25.881679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a58658-7c63-4551-94a2-8a8b5cf72446 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:25.887634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4435b2-a0cb-4882-a16a-e00c67b13e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:25.888313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4435b2-a0cb-4882-a16a-e00c67b13e83 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:25.888839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4435b2-a0cb-4882-a16a-e00c67b13e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:25.889369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4435b2-a0cb-4882-a16a-e00c67b13e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:25.890116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4435b2-a0cb-4882-a16a-e00c67b13e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:25.892248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4435b2-a0cb-4882-a16a-e00c67b13e83 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:25.927968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1252/2476] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:25 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:25.929461 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57c734-05ce-4d9d-8de2-c4a5d5bde581 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:25.940273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57c734-05ce-4d9d-8de2-c4a5d5bde581 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:25.998286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d6c3b-79da-4811-be32-c9c8a6e1ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:25.998903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d6c3b-79da-4811-be32-c9c8a6e1ddf5 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:25.999184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d6c3b-79da-4811-be32-c9c8a6e1ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:25.999456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d6c3b-79da-4811-be32-c9c8a6e1ddf5 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:26.000099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d6c3b-79da-4811-be32-c9c8a6e1ddf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:26.002061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d6c3b-79da-4811-be32-c9c8a6e1ddf5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:26.027924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1225/2477] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:25 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:26.032024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4435b2-a0cb-4882-a16a-e00c67b13e83 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:26.042534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4435b2-a0cb-4882-a16a-e00c67b13e83 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:26.049347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eb1b1-6c99-430c-bd91-65283d1da2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:26.050387 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eb1b1-6c99-430c-bd91-65283d1da2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:26.050882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eb1b1-6c99-430c-bd91-65283d1da2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:26.051195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eb1b1-6c99-430c-bd91-65283d1da2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:26.051869 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eb1b1-6c99-430c-bd91-65283d1da2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:26.053498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eb1b1-6c99-430c-bd91-65283d1da2b9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:26.064590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1253/2478] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:26.069026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d6c3b-79da-4811-be32-c9c8a6e1ddf5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:26.079615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d6c3b-79da-4811-be32-c9c8a6e1ddf5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:26.084462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6927d-c328-48a3-b9c2-ebce509b2432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:26.085122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6927d-c328-48a3-b9c2-ebce509b2432 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:26.085408 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6927d-c328-48a3-b9c2-ebce509b2432 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:26.085728 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6927d-c328-48a3-b9c2-ebce509b2432 admin admin] PATH_INFO: `/v3/projects/b512b4029fc849b4b2d153f23867acd7/users/8c51eb38242f4aa5abffc33e584281e2/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:26.102197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6927d-c328-48a3-b9c2-ebce509b2432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b512b4029fc849b4b2d153f23867acd7, user_id=8c51eb38242f4aa5abffc33e584281e2, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:26.104911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6927d-c328-48a3-b9c2-ebce509b2432 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:26.119467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1226/2479] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:26 2022] PUT /identity/v3/projects/b512b4029fc849b4b2d153f23867acd7/users/8c51eb38242f4aa5abffc33e584281e2/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:26.124173 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eb1b1-6c99-430c-bd91-65283d1da2b9 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:26.134815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eb1b1-6c99-430c-bd91-65283d1da2b9 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:26.192963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0e106-6220-4909-a020-fe8b57017ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:26.193578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0e106-6220-4909-a020-fe8b57017ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:26.193888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0e106-6220-4909-a020-fe8b57017ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:26.194161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0e106-6220-4909-a020-fe8b57017ae9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:26.194810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0e106-6220-4909-a020-fe8b57017ae9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:26.196477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0e106-6220-4909-a020-fe8b57017ae9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:26.209678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1254/2480] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:26.211290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6927d-c328-48a3-b9c2-ebce509b2432 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:26.221695 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6927d-c328-48a3-b9c2-ebce509b2432 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:26.226204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:26.226842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:26.227120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:26.227391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] PATH_INFO: `/v3/projects/b512b4029fc849b4b2d153f23867acd7/users/8c51eb38242f4aa5abffc33e584281e2/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:26.229270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b512b4029fc849b4b2d153f23867acd7, user_id=8c51eb38242f4aa5abffc33e584281e2, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:26.232045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:26.247291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1227/2481] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:26 2022] PUT /identity/v3/projects/b512b4029fc849b4b2d153f23867acd7/users/8c51eb38242f4aa5abffc33e584281e2/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:26.927546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe121-9d10-4010-8286-403ca6b13d37 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:26.928060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe121-9d10-4010-8286-403ca6b13d37 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:26.928217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe121-9d10-4010-8286-403ca6b13d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:26.956237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-15ffe121-9d10-4010-8286-403ca6b13d37 None None] MFA Rules not processed for user `8c51eb38242f4aa5abffc33e584281e2`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:26.957431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-15ffe121-9d10-4010-8286-403ca6b13d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:27.036019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1255/2482] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:27.047972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:27.049720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:27.094189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:27.109358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec764a4-c884-4d35-a4d8-c6c131b8094b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:27.112891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:27.113489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:27.113797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:27.114067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:27.153033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffe121-9d10-4010-8286-403ca6b13d37 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:27.173499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:27.175015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:27.183922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1228/2483] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:27.184528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:27.210880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffe121-9d10-4010-8286-403ca6b13d37 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:27.215336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:27.216184 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:27.216464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:27.216734 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:27.218449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:27.227866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:27.229311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:27.238336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1256/2484] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:28.572696 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:28.574216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:28.588250 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8688119-8674-4b36-90bd-a2987e629144 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:28.592845 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76858b21-1bdf-451b-a81f-b95c98f3f65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:28.595952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858b21-1bdf-451b-a81f-b95c98f3f65d service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:28.595952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858b21-1bdf-451b-a81f-b95c98f3f65d service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:28.595952 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76858b21-1bdf-451b-a81f-b95c98f3f65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:28.605938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76858b21-1bdf-451b-a81f-b95c98f3f65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:28.607357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76858b21-1bdf-451b-a81f-b95c98f3f65d service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:28.616972 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1229/2485] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:29.840066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:29.841618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:29.916323 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927eac07-eb8e-49a5-b242-c756b1fe6231 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:29.923197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:29.923952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:29.924248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:29.924494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] PATH_INFO: `/v3/users/c535b1b4c45b47c3a97fa698bbc772bc` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:29.926011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c535b1b4c45b47c3a97fa698bbc772bc)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:29.928385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:29.989525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c535b1b4c45b47c3a97fa698bbc772bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:29.998834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c535b1b4c45b47c3a97fa698bbc772bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:29.999626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c535b1b4c45b47c3a97fa698bbc772bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:30.021139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c535b1b4c45b47c3a97fa698bbc772bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:30.030021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c535b1b4c45b47c3a97fa698bbc772bc'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:30.036142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1257/2486] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:29 2022] DELETE /identity/v3/users/c535b1b4c45b47c3a97fa698bbc772bc => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:30.042232 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f072a-88a0-4485-b0d5-ec6178e34dca None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.042966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f072a-88a0-4485-b0d5-ec6178e34dca None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.043260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f072a-88a0-4485-b0d5-ec6178e34dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.066046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-788f072a-88a0-4485-b0d5-ec6178e34dca None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:30.067800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-788f072a-88a0-4485-b0d5-ec6178e34dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:30.152100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1230/2487] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:30.152217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:30.152217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:30.222618 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f270ff-a3c2-4764-8a80-77e09736c36a admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:30.226399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470aee8b-9a4a-4d55-9361-476f5029fff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:30.227011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470aee8b-9a4a-4d55-9361-476f5029fff1 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.227288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470aee8b-9a4a-4d55-9361-476f5029fff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.227563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470aee8b-9a4a-4d55-9361-476f5029fff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.234079 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:30.239101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470aee8b-9a4a-4d55-9361-476f5029fff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:30.240909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470aee8b-9a4a-4d55-9361-476f5029fff1 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:30.252793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1258/2488] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:30.524787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f072a-88a0-4485-b0d5-ec6178e34dca None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:30.526888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-788f072a-88a0-4485-b0d5-ec6178e34dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:30.540341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f072a-88a0-4485-b0d5-ec6178e34dca None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:30.545674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:30.546390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.546677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.547050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] PATH_INFO: `/v3/projects/f5dd48d04bc44304b41c9c51a1f1bac5` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.548472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5dd48d04bc44304b41c9c51a1f1bac5)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:30.550545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:30.622430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5dd48d04bc44304b41c9c51a1f1bac5', 'request_id': 'req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:30.628073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5dd48d04bc44304b41c9c51a1f1bac5', 'request_id': 'req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:30.634815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5dd48d04bc44304b41c9c51a1f1bac5', 'request_id': 'req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:30.648497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] The token cache is being invalidate because project f5dd48d04bc44304b41c9c51a1f1bac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:30.648928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:30.649737 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1f7ea68a-34ca-4bbc-9fd6-f044d28d8464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:30.651523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1231/2489] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:30 2022] DELETE /identity/v3/projects/f5dd48d04bc44304b41c9c51a1f1bac5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:30.656376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d2b72-b4c4-4e76-aab6-f1de8af315ac None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.656663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d2b72-b4c4-4e76-aab6-f1de8af315ac None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.656942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0d2b72-b4c4-4e76-aab6-f1de8af315ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.676741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3f0d2b72-b4c4-4e76-aab6-f1de8af315ac None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:30.678311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3f0d2b72-b4c4-4e76-aab6-f1de8af315ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:30.739959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1259/2490] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:30.748665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3287189-af6f-4562-96d1-6b67779fcd5d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.748959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3287189-af6f-4562-96d1-6b67779fcd5d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.749241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3287189-af6f-4562-96d1-6b67779fcd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.757802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:30.769799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-a3287189-af6f-4562-96d1-6b67779fcd5d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:30.771131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a3287189-af6f-4562-96d1-6b67779fcd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:30.779123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1232/2491] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:30.783266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d2b72-b4c4-4e76-aab6-f1de8af315ac None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:30.792943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0d2b72-b4c4-4e76-aab6-f1de8af315ac None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:30.797853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:30.798518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.798803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.799077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.799723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:30.800027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:30.801655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:30.813858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1260/2492] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:25:30.817869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3287189-af6f-4562-96d1-6b67779fcd5d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:30.827574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3287189-af6f-4562-96d1-6b67779fcd5d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:30.832236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:30.832830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.833111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.833383 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.834039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:30.835989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:30.867632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1233/2493] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:30 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:30.868968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:30.873932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:30.875606 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:30.880151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798dc525-1c7b-43fc-a72b-042f01862540 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:30.936575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa5f35-7d64-42e9-beba-8ce3de876f66 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:30.940622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:30.941242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.941691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.941994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.943745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:30.948115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b81204-9656-411b-a785-e38c1ad65f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:30.948725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81204-9656-411b-a785-e38c1ad65f0c admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.949025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81204-9656-411b-a785-e38c1ad65f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.949277 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81204-9656-411b-a785-e38c1ad65f0c admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.949951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b81204-9656-411b-a785-e38c1ad65f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:30.952420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b81204-9656-411b-a785-e38c1ad65f0c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:30.956497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:30.957971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:30.968611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1234/2494] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:30.977530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1261/2495] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:30 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:30.979455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:30.990210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c73a7-9c68-4477-baff-3193f89ae835 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:30.996297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89018733-a87d-4c37-980e-6b2fa3a0121d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:30.996708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89018733-a87d-4c37-980e-6b2fa3a0121d admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:30.997066 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89018733-a87d-4c37-980e-6b2fa3a0121d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:30.997412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89018733-a87d-4c37-980e-6b2fa3a0121d admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:30.998175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89018733-a87d-4c37-980e-6b2fa3a0121d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.000225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89018733-a87d-4c37-980e-6b2fa3a0121d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.020549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1235/2496] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:31.021971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b81204-9656-411b-a785-e38c1ad65f0c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.048507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b81204-9656-411b-a785-e38c1ad65f0c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.048507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05a62a-c9f6-450b-a06e-0561995f9583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.048507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05a62a-c9f6-450b-a06e-0561995f9583 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.048507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05a62a-c9f6-450b-a06e-0561995f9583 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.048948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05a62a-c9f6-450b-a06e-0561995f9583 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.048948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05a62a-c9f6-450b-a06e-0561995f9583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.048948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05a62a-c9f6-450b-a06e-0561995f9583 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.060320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1262/2497] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:31.064539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89018733-a87d-4c37-980e-6b2fa3a0121d admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.075135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89018733-a87d-4c37-980e-6b2fa3a0121d admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.079798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e7f8a-2bc4-4e16-842c-b09914f878cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.080449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e7f8a-2bc4-4e16-842c-b09914f878cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.080733 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e7f8a-2bc4-4e16-842c-b09914f878cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.081012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e7f8a-2bc4-4e16-842c-b09914f878cb admin admin] PATH_INFO: `/v3/projects/3d4e93b08fb7446080ab340b9228293d/users/8a3429260c224dbcabe56043c47a4dcb/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.096682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e7f8a-2bc4-4e16-842c-b09914f878cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4e93b08fb7446080ab340b9228293d, user_id=8a3429260c224dbcabe56043c47a4dcb, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.099467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e7f8a-2bc4-4e16-842c-b09914f878cb admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.113233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1236/2498] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:31 2022] PUT /identity/v3/projects/3d4e93b08fb7446080ab340b9228293d/users/8a3429260c224dbcabe56043c47a4dcb/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:31.124394 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971b5bd-51ba-4257-a66d-6638a939964f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.124675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971b5bd-51ba-4257-a66d-6638a939964f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.124942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971b5bd-51ba-4257-a66d-6638a939964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.153433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3971b5bd-51ba-4257-a66d-6638a939964f None None] MFA Rules not processed for user `8a3429260c224dbcabe56043c47a4dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:31.154691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3971b5bd-51ba-4257-a66d-6638a939964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:31.215532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:31.248239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1263/2499] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:31.250174 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e7f8a-2bc4-4e16-842c-b09914f878cb admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.321884 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e7f8a-2bc4-4e16-842c-b09914f878cb admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.327124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d3871-1c88-45b4-8cd6-9d3f0045b03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.327739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3871-1c88-45b4-8cd6-9d3f0045b03a admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.328021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3871-1c88-45b4-8cd6-9d3f0045b03a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.328290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3871-1c88-45b4-8cd6-9d3f0045b03a admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.328937 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d3871-1c88-45b4-8cd6-9d3f0045b03a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.331026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d3871-1c88-45b4-8cd6-9d3f0045b03a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.365438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1237/2500] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:31 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:31.370009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971b5bd-51ba-4257-a66d-6638a939964f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.381264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971b5bd-51ba-4257-a66d-6638a939964f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.462931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.464203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.464661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.465106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.466033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.468754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.513068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1264/2501] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:31 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:31.517049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d3871-1c88-45b4-8cd6-9d3f0045b03a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.532178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d3871-1c88-45b4-8cd6-9d3f0045b03a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.538357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f674c8-71b0-430b-99b9-694204c94f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.539083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f674c8-71b0-430b-99b9-694204c94f76 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.539367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f674c8-71b0-430b-99b9-694204c94f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.539646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f674c8-71b0-430b-99b9-694204c94f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.540817 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f674c8-71b0-430b-99b9-694204c94f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.545815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f674c8-71b0-430b-99b9-694204c94f76 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.545974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.545974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:31.561246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1238/2502] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:31.562183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f4c4-644e-4ec5-9420-4f6fd46c214b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.562501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f674c8-71b0-430b-99b9-694204c94f76 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.567219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.567856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.568137 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.568411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] PATH_INFO: `/v3/users/1500cbd765c74fa488e678dda25f5e58` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.569582 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1500cbd765c74fa488e678dda25f5e58)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.571544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.572933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f674c8-71b0-430b-99b9-694204c94f76 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.578145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d307995-7c1f-4f8a-b403-f59dec036a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.578789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307995-7c1f-4f8a-b403-f59dec036a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.579074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307995-7c1f-4f8a-b403-f59dec036a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.579349 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d307995-7c1f-4f8a-b403-f59dec036a01 admin admin] PATH_INFO: `/v3/projects/60d9a08189db4a79aa6102c60ae985a9/users/37247b7cde0b4a6db91c12a4fc52c839/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.594882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d307995-7c1f-4f8a-b403-f59dec036a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d9a08189db4a79aa6102c60ae985a9, user_id=37247b7cde0b4a6db91c12a4fc52c839, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.597779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d307995-7c1f-4f8a-b403-f59dec036a01 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.600482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:31.613710 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1239/2503] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:31 2022] PUT /identity/v3/projects/60d9a08189db4a79aa6102c60ae985a9/users/37247b7cde0b4a6db91c12a4fc52c839/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:31.617747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d307995-7c1f-4f8a-b403-f59dec036a01 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.623721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1500cbd765c74fa488e678dda25f5e58'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:31.631876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1500cbd765c74fa488e678dda25f5e58'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:31.632922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1500cbd765c74fa488e678dda25f5e58'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:31.646115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1500cbd765c74fa488e678dda25f5e58'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:31.654015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1500cbd765c74fa488e678dda25f5e58'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:31.660152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1265/2504] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:31 2022] DELETE /identity/v3/users/1500cbd765c74fa488e678dda25f5e58 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:31.683921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d307995-7c1f-4f8a-b403-f59dec036a01 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.757517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48353e-a56d-45c4-8f06-b772d43a0002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.758155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48353e-a56d-45c4-8f06-b772d43a0002 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.758434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48353e-a56d-45c4-8f06-b772d43a0002 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.758763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48353e-a56d-45c4-8f06-b772d43a0002 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.759397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48353e-a56d-45c4-8f06-b772d43a0002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.761651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48353e-a56d-45c4-8f06-b772d43a0002 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.771776 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1240/2505] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:31.776433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.777937 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:31.791014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477945f-940b-4702-8e23-9fa109c76615 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.795884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.796549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.796829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.797105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] PATH_INFO: `/v3/projects/60d9a08189db4a79aa6102c60ae985a9/users/37247b7cde0b4a6db91c12a4fc52c839/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.799114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d9a08189db4a79aa6102c60ae985a9, user_id=37247b7cde0b4a6db91c12a4fc52c839, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.801825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.812308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1266/2506] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:31 2022] PUT /identity/v3/projects/60d9a08189db4a79aa6102c60ae985a9/users/37247b7cde0b4a6db91c12a4fc52c839/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:31.824732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfbf1c-9251-40c6-9afe-d3ec53219f37 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.825094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfbf1c-9251-40c6-9afe-d3ec53219f37 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.825314 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcfbf1c-9251-40c6-9afe-d3ec53219f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.842119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.844338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:31.857394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-6dcfbf1c-9251-40c6-9afe-d3ec53219f37 None None] MFA Rules not processed for user `37247b7cde0b4a6db91c12a4fc52c839`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:31.858672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6dcfbf1c-9251-40c6-9afe-d3ec53219f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:31.894243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:31.919548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50905fc-d36f-45ee-aff0-3b9264a532f4 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:31.924443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:31.925067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:31.925577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:31.925758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] PATH_INFO: `/v3/projects/d030d7bb8d92404e9083b3847fcb0487` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:31.926772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d030d7bb8d92404e9083b3847fcb0487)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:31.928699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:31.930769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1241/2507] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:31.942646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfbf1c-9251-40c6-9afe-d3ec53219f37 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:31.944080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-6dcfbf1c-9251-40c6-9afe-d3ec53219f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:31.975244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:31.982879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd030d7bb8d92404e9083b3847fcb0487', 'request_id': 'req-3203abee-4c45-4962-8ac5-5dc79cda7991'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:31.988599 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd030d7bb8d92404e9083b3847fcb0487', 'request_id': 'req-3203abee-4c45-4962-8ac5-5dc79cda7991'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:31.996315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd030d7bb8d92404e9083b3847fcb0487', 'request_id': 'req-3203abee-4c45-4962-8ac5-5dc79cda7991'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:32.008280 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcfbf1c-9251-40c6-9afe-d3ec53219f37 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:32.008643 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] The token cache is being invalidate because project d030d7bb8d92404e9083b3847fcb0487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:32.008932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:32.009563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:32.020206 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1267/2508] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:31 2022] DELETE /identity/v3/projects/d030d7bb8d92404e9083b3847fcb0487 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:32.075463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:32.076127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:32.076355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:32.076640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:32.078684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:32.160745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:32.162271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:32.172868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1242/2509] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:32.252558 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:32.253993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:32.266473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203abee-4c45-4962-8ac5-5dc79cda7991 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:32.269996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c818c-1781-477a-abf3-22ef16def431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:32.270585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c818c-1781-477a-abf3-22ef16def431 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:32.270855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c818c-1781-477a-abf3-22ef16def431 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:32.271119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c818c-1781-477a-abf3-22ef16def431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:32.272745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-dd4c818c-1781-477a-abf3-22ef16def431 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:32.330528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c818c-1781-477a-abf3-22ef16def431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:32.331889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c818c-1781-477a-abf3-22ef16def431 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:32.341120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1268/2510] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:42.168235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=60 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:42.188342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:42.189769 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:42.253798 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4236cd6-70cb-478b-8a60-038d66ce41de service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:42.258242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:42.258855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:42.259134 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:42.259406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] PATH_INFO: `/v3/users/8a3429260c224dbcabe56043c47a4dcb` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:42.260491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3429260c224dbcabe56043c47a4dcb)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:42.262568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:42.307834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3429260c224dbcabe56043c47a4dcb'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:42.315485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3429260c224dbcabe56043c47a4dcb'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:42.316116 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3429260c224dbcabe56043c47a4dcb'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:42.328115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3429260c224dbcabe56043c47a4dcb'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:42.334872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3429260c224dbcabe56043c47a4dcb'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:42.336590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:42.341561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1243/2511] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:42 2022] DELETE /identity/v3/users/8a3429260c224dbcabe56043c47a4dcb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:42.346113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d300-53d5-46ac-8918-4b035596d629 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:42.346397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d300-53d5-46ac-8918-4b035596d629 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:42.346670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04d300-53d5-46ac-8918-4b035596d629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:42.366022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-8f04d300-53d5-46ac-8918-4b035596d629 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:42.367209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8f04d300-53d5-46ac-8918-4b035596d629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:42.425621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1269/2512] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:42.435964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:42.437526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:42.494029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abe266-60dc-4482-a0d7-908498ea89ff admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:42.498212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211bae4-4f95-4d79-ac67-cebe5e37d5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:42.498812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211bae4-4f95-4d79-ac67-cebe5e37d5ec service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:42.499095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211bae4-4f95-4d79-ac67-cebe5e37d5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:42.499374 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211bae4-4f95-4d79-ac67-cebe5e37d5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:42.508292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211bae4-4f95-4d79-ac67-cebe5e37d5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:42.509764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211bae4-4f95-4d79-ac67-cebe5e37d5ec service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:42.519159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1244/2513] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:43.038356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04d300-53d5-46ac-8918-4b035596d629 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:43.039874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8f04d300-53d5-46ac-8918-4b035596d629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:43.051882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04d300-53d5-46ac-8918-4b035596d629 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:43.056556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:43.057248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:43.057591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:43.057927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] PATH_INFO: `/v3/projects/3d4e93b08fb7446080ab340b9228293d` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:43.059120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4e93b08fb7446080ab340b9228293d)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:43.061041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:43.114750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4e93b08fb7446080ab340b9228293d', 'request_id': 'req-cc351efe-cd1f-4ee3-b209-9932e384263f'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.120902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4e93b08fb7446080ab340b9228293d', 'request_id': 'req-cc351efe-cd1f-4ee3-b209-9932e384263f'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.127328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4e93b08fb7446080ab340b9228293d', 'request_id': 'req-cc351efe-cd1f-4ee3-b209-9932e384263f'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.140503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] The token cache is being invalidate because project 3d4e93b08fb7446080ab340b9228293d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:43.140831 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.141441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.143515 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1270/2514] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:43 2022] DELETE /identity/v3/projects/3d4e93b08fb7446080ab340b9228293d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:43.147707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211bae4-4f95-4d79-ac67-cebe5e37d5ec service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:43.158247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211bae4-4f95-4d79-ac67-cebe5e37d5ec service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:43.176067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:43.220322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:43.220962 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:43.221244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:43.221517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] PATH_INFO: `/v3/users/37247b7cde0b4a6db91c12a4fc52c839` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:43.222746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37247b7cde0b4a6db91c12a4fc52c839)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:43.224693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:43.275927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37247b7cde0b4a6db91c12a4fc52c839'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.284104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37247b7cde0b4a6db91c12a4fc52c839'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.284753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37247b7cde0b4a6db91c12a4fc52c839'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.297157 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37247b7cde0b4a6db91c12a4fc52c839'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.304160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8b25a254-c897-42e8-82e8-1471bef7f6b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37247b7cde0b4a6db91c12a4fc52c839'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.310650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1245/2515] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:43 2022] DELETE /identity/v3/users/37247b7cde0b4a6db91c12a4fc52c839 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:43.687705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:43.689217 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:43.705214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc351efe-cd1f-4ee3-b209-9932e384263f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:43.773878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:43.774626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:43.775249 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:43.775249 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] PATH_INFO: `/v3/projects/60d9a08189db4a79aa6102c60ae985a9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:43.776691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d9a08189db4a79aa6102c60ae985a9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:43.778789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:43.811121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:43.829839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d9a08189db4a79aa6102c60ae985a9', 'request_id': 'req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.835973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d9a08189db4a79aa6102c60ae985a9', 'request_id': 'req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.845444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d9a08189db4a79aa6102c60ae985a9', 'request_id': 'req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.860398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] The token cache is being invalidate because project 60d9a08189db4a79aa6102c60ae985a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:43.860772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.861485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d3ae5406-5f21-40f7-9a90-70f04bdc32b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:43.863111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1271/2516] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:43 2022] DELETE /identity/v3/projects/60d9a08189db4a79aa6102c60ae985a9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:43.868052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed11f99-8dce-47b7-847e-a0c0fd36df35 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:43.868460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed11f99-8dce-47b7-847e-a0c0fd36df35 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:43.869063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed11f99-8dce-47b7-847e-a0c0fd36df35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:43.890049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7ed11f99-8dce-47b7-847e-a0c0fd36df35 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:43.891267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7ed11f99-8dce-47b7-847e-a0c0fd36df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:43.955820 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1246/2517] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:43.963442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731066a3-81fe-4cbf-9747-4713ac24b197 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:43.963733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731066a3-81fe-4cbf-9747-4713ac24b197 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:43.964051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731066a3-81fe-4cbf-9747-4713ac24b197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:43.985423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-731066a3-81fe-4cbf-9747-4713ac24b197 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:43.986660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-731066a3-81fe-4cbf-9747-4713ac24b197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:43.995034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1272/2518] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:43.999856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed11f99-8dce-47b7-847e-a0c0fd36df35 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.010675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed11f99-8dce-47b7-847e-a0c0fd36df35 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.017253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.018373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.018373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.018602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.019072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:44.019381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.021006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.034231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1247/2519] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:25:44.038705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731066a3-81fe-4cbf-9747-4713ac24b197 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.048571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731066a3-81fe-4cbf-9747-4713ac24b197 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.053264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.053980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.054180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.054456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.055116 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.057017 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.083305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1273/2520] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:44 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:44.087435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.098976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e089f-d345-4532-9a7d-9744f7f2d4bf admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.126930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:44.142875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.144795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:44.164079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a27e93-4121-49d2-9d8d-814ae941088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.164693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a27e93-4121-49d2-9d8d-814ae941088a admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.164976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a27e93-4121-49d2-9d8d-814ae941088a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.165251 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a27e93-4121-49d2-9d8d-814ae941088a admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.165936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a27e93-4121-49d2-9d8d-814ae941088a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.168479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a27e93-4121-49d2-9d8d-814ae941088a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.190565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1248/2521] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:44 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:44.194189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a27e93-4121-49d2-9d8d-814ae941088a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.204041 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a27e93-4121-49d2-9d8d-814ae941088a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.208918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3f714-1bb1-4d10-b0b4-5a4d89d2c7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.209421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3f714-1bb1-4d10-b0b4-5a4d89d2c7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.209732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3f714-1bb1-4d10-b0b4-5a4d89d2c7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.210008 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3f714-1bb1-4d10-b0b4-5a4d89d2c7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.210638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3f714-1bb1-4d10-b0b4-5a4d89d2c7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.212628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3f714-1bb1-4d10-b0b4-5a4d89d2c7d9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.216467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479969a0-9ab9-480e-b7cc-43e14b2dbf23 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.221193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.221940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.222279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.222616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] PATH_INFO: `/v3/users/0b48e5e48d2b442ba3dd6ce0dab533ee` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.226544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1249/2522] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:44.226692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b48e5e48d2b442ba3dd6ce0dab533ee)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.226692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.227352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3f714-1bb1-4d10-b0b4-5a4d89d2c7d9 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.237830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3f714-1bb1-4d10-b0b4-5a4d89d2c7d9 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.242464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44741b-fc29-403e-9e32-11feac7e737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.243060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44741b-fc29-403e-9e32-11feac7e737f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.243329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44741b-fc29-403e-9e32-11feac7e737f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.243599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad44741b-fc29-403e-9e32-11feac7e737f admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.244477 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44741b-fc29-403e-9e32-11feac7e737f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.246269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad44741b-fc29-403e-9e32-11feac7e737f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.257061 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1250/2523] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:44.261393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44741b-fc29-403e-9e32-11feac7e737f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.270854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad44741b-fc29-403e-9e32-11feac7e737f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.272654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b48e5e48d2b442ba3dd6ce0dab533ee'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.275456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.276105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.276366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.277177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] PATH_INFO: `/v3/projects/48314ec93bb047728923f740a35d7126/users/9594569f26164f74a99aabb787d9df2c/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.281690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b48e5e48d2b442ba3dd6ce0dab533ee'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.282451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b48e5e48d2b442ba3dd6ce0dab533ee'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.291939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=48314ec93bb047728923f740a35d7126, user_id=9594569f26164f74a99aabb787d9df2c, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.295295 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.295692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b48e5e48d2b442ba3dd6ce0dab533ee'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.302614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-465670b9-b13e-4b1e-a4c6-c71dd1e2294e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b48e5e48d2b442ba3dd6ce0dab533ee'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.308818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1274/2524] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:44 2022] DELETE /identity/v3/users/0b48e5e48d2b442ba3dd6ce0dab533ee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:44.309509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1251/2525] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:44 2022] PUT /identity/v3/projects/48314ec93bb047728923f740a35d7126/users/9594569f26164f74a99aabb787d9df2c/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:44.319557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24743a-17a5-42bf-a15c-d025a11248e6 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.319846 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24743a-17a5-42bf-a15c-d025a11248e6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.320127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24743a-17a5-42bf-a15c-d025a11248e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.328848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:44.351690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5b24743a-17a5-42bf-a15c-d025a11248e6 None None] MFA Rules not processed for user `9594569f26164f74a99aabb787d9df2c`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:44.352936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5b24743a-17a5-42bf-a15c-d025a11248e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:44.421548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1275/2526] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:44.426080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.427858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:44.488941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24743a-17a5-42bf-a15c-d025a11248e6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.490464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5b24743a-17a5-42bf-a15c-d025a11248e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:44.495449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b55b46-e0a8-4ac4-a2d3-a08cf278ff7e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.561674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eb170-8c54-4ad3-9097-645340c07242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.562386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eb170-8c54-4ad3-9097-645340c07242 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.562772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eb170-8c54-4ad3-9097-645340c07242 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.563054 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24743a-17a5-42bf-a15c-d025a11248e6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.563177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7eb170-8c54-4ad3-9097-645340c07242 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.563866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eb170-8c54-4ad3-9097-645340c07242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.565866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7eb170-8c54-4ad3-9097-645340c07242 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.567731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.568359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.568644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.568916 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] PATH_INFO: `/v3/projects/017865427104453898d35dcd902cf497` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.570077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=017865427104453898d35dcd902cf497)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.571976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.595085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1252/2527] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:44 2022] POST /identity/v3/projects => generated 371 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:44.598959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eb170-8c54-4ad3-9097-645340c07242 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.604072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:44.625356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017865427104453898d35dcd902cf497', 'request_id': 'req-aae73a72-b755-4e74-88ba-d99272d931ef'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.631102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017865427104453898d35dcd902cf497', 'request_id': 'req-aae73a72-b755-4e74-88ba-d99272d931ef'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.637848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017865427104453898d35dcd902cf497', 'request_id': 'req-aae73a72-b755-4e74-88ba-d99272d931ef'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.650865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] The token cache is being invalidate because project 017865427104453898d35dcd902cf497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:44.651164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.652802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.654965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1276/2528] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:44 2022] DELETE /identity/v3/projects/017865427104453898d35dcd902cf497 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:44.658889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.660242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:44.665190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7eb170-8c54-4ad3-9097-645340c07242 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.688384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:44.722403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.723262 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.723835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.723994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.724630 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.727608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.730309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae73a72-b755-4e74-88ba-d99272d931ef admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.735309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.735879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.736154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.736432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] PATH_INFO: `/v3/users/8c51eb38242f4aa5abffc33e584281e2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.737553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c51eb38242f4aa5abffc33e584281e2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.739493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.749801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1253/2529] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:44 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:44.753948 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.755579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:44.768255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b23fc-cb01-498a-8ffe-1a5f8ef43e1e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.773336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2fbf3-eae3-4e50-a0ea-6b91c4f78e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.774229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2fbf3-eae3-4e50-a0ea-6b91c4f78e4b admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.774642 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2fbf3-eae3-4e50-a0ea-6b91c4f78e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.775018 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c2fbf3-eae3-4e50-a0ea-6b91c4f78e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.775742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2fbf3-eae3-4e50-a0ea-6b91c4f78e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.778023 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c2fbf3-eae3-4e50-a0ea-6b91c4f78e4b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.788378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1254/2530] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:44.791441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c51eb38242f4aa5abffc33e584281e2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.792169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2fbf3-eae3-4e50-a0ea-6b91c4f78e4b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.799959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c51eb38242f4aa5abffc33e584281e2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.800656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c51eb38242f4aa5abffc33e584281e2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.804263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c2fbf3-eae3-4e50-a0ea-6b91c4f78e4b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.810296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f101fde-c949-4cc3-b7be-6c04dbbf72cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.811505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f101fde-c949-4cc3-b7be-6c04dbbf72cb admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.811789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f101fde-c949-4cc3-b7be-6c04dbbf72cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.812068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f101fde-c949-4cc3-b7be-6c04dbbf72cb admin admin] PATH_INFO: `/v3/projects/59c28df0248e42ee97eb553dbbb337fd/users/1f7ab5a580bd436cb38e8f77f887387e/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.814245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c51eb38242f4aa5abffc33e584281e2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.822018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c51eb38242f4aa5abffc33e584281e2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:44.829529 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1277/2531] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:44 2022] DELETE /identity/v3/users/8c51eb38242f4aa5abffc33e584281e2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:44.829891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f101fde-c949-4cc3-b7be-6c04dbbf72cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c28df0248e42ee97eb553dbbb337fd, user_id=1f7ab5a580bd436cb38e8f77f887387e, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.832638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f101fde-c949-4cc3-b7be-6c04dbbf72cb admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.849057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1255/2532] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:44 2022] PUT /identity/v3/projects/59c28df0248e42ee97eb553dbbb337fd/users/1f7ab5a580bd436cb38e8f77f887387e/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:44.853129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:44.854614 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:44.919808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f14397-0b0e-4b08-a7df-56f8c10c6eb3 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:44.967143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:44.980149 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:44.980782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:44.981026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:44.981287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:44.981975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:44.983609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:44.994039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1278/2533] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:44.998970 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f101fde-c949-4cc3-b7be-6c04dbbf72cb admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.009729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.012948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.016949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f101fde-c949-4cc3-b7be-6c04dbbf72cb admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.026074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870c40-7b08-448d-aa76-2f5fd9da115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.026821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870c40-7b08-448d-aa76-2f5fd9da115c admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.027109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870c40-7b08-448d-aa76-2f5fd9da115c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.027381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870c40-7b08-448d-aa76-2f5fd9da115c admin admin] PATH_INFO: `/v3/projects/59c28df0248e42ee97eb553dbbb337fd/users/1f7ab5a580bd436cb38e8f77f887387e/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.029364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870c40-7b08-448d-aa76-2f5fd9da115c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c28df0248e42ee97eb553dbbb337fd, user_id=1f7ab5a580bd436cb38e8f77f887387e, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.032088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870c40-7b08-448d-aa76-2f5fd9da115c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.033359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907e514-ca9a-4e7c-9d2c-48365a887f6d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.039880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.040572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.040925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.041128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] PATH_INFO: `/v3/projects/b512b4029fc849b4b2d153f23867acd7` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.042456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b512b4029fc849b4b2d153f23867acd7)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.043870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1256/2534] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:44 2022] PUT /identity/v3/projects/59c28df0248e42ee97eb553dbbb337fd/users/1f7ab5a580bd436cb38e8f77f887387e/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:45.044416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.055076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442e8be-0995-44f6-8300-1cfd2b705ab0 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.055380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442e8be-0995-44f6-8300-1cfd2b705ab0 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.055659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442e8be-0995-44f6-8300-1cfd2b705ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.084827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-0442e8be-0995-44f6-8300-1cfd2b705ab0 None None] MFA Rules not processed for user `1f7ab5a580bd436cb38e8f77f887387e`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:45.086086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0442e8be-0995-44f6-8300-1cfd2b705ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.098775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b512b4029fc849b4b2d153f23867acd7', 'request_id': 'req-3da7d61c-307b-40ff-a420-4b615cfe034c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.104695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b512b4029fc849b4b2d153f23867acd7', 'request_id': 'req-3da7d61c-307b-40ff-a420-4b615cfe034c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.113303 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b512b4029fc849b4b2d153f23867acd7', 'request_id': 'req-3da7d61c-307b-40ff-a420-4b615cfe034c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.127294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] The token cache is being invalidate because project b512b4029fc849b4b2d153f23867acd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:45.127588 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.128238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3da7d61c-307b-40ff-a420-4b615cfe034c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.129929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1279/2535] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:45 2022] DELETE /identity/v3/projects/b512b4029fc849b4b2d153f23867acd7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:45.132053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:45.134768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abdb50-c763-453f-a56b-dab78e3c1c26 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.135053 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abdb50-c763-453f-a56b-dab78e3c1c26 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.135328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abdb50-c763-453f-a56b-dab78e3c1c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.155625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-62abdb50-c763-453f-a56b-dab78e3c1c26 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:45.156884 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-62abdb50-c763-453f-a56b-dab78e3c1c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.161669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1257/2536] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:45.173409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442e8be-0995-44f6-8300-1cfd2b705ab0 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.174806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0442e8be-0995-44f6-8300-1cfd2b705ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.223191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1280/2537] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:45.231174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88df92-a08e-419e-81bb-3d821e170e83 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.231459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88df92-a08e-419e-81bb-3d821e170e83 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.231739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88df92-a08e-419e-81bb-3d821e170e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.238662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442e8be-0995-44f6-8300-1cfd2b705ab0 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.242407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61dedd-0ff2-4048-8884-d0b48346e9ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.243011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61dedd-0ff2-4048-8884-d0b48346e9ef service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.243348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61dedd-0ff2-4048-8884-d0b48346e9ef service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.243885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61dedd-0ff2-4048-8884-d0b48346e9ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.252106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ef88df92-a08e-419e-81bb-3d821e170e83 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:45.253251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ef88df92-a08e-419e-81bb-3d821e170e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.253594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61dedd-0ff2-4048-8884-d0b48346e9ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.255069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61dedd-0ff2-4048-8884-d0b48346e9ef service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.261580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1281/2538] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:25:45.264690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1258/2539] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:45.265273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88df92-a08e-419e-81bb-3d821e170e83 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.274426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88df92-a08e-419e-81bb-3d821e170e83 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.279164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.279758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.280040 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.280313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.280951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:45.281374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.283182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.299560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1282/2540] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:25:45.300915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61dedd-0ff2-4048-8884-d0b48346e9ef service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.312176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61dedd-0ff2-4048-8884-d0b48346e9ef service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.316905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557535a-ac8f-48ba-a476-81b18dfe2e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.317531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557535a-ac8f-48ba-a476-81b18dfe2e18 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.317856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557535a-ac8f-48ba-a476-81b18dfe2e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.318131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557535a-ac8f-48ba-a476-81b18dfe2e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.318772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557535a-ac8f-48ba-a476-81b18dfe2e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.320675 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557535a-ac8f-48ba-a476-81b18dfe2e18 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.351898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1259/2541] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:45.353193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.364527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440a48-bfdd-41eb-a1cc-78801c718eb3 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.415834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:45.431830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c42878-5148-49b8-95f3-e1722aa950e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.432439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c42878-5148-49b8-95f3-e1722aa950e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.433322 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c42878-5148-49b8-95f3-e1722aa950e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.433600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c42878-5148-49b8-95f3-e1722aa950e8 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.434260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c42878-5148-49b8-95f3-e1722aa950e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.436157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c42878-5148-49b8-95f3-e1722aa950e8 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.458517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1283/2542] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:45.462508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557535a-ac8f-48ba-a476-81b18dfe2e18 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.473090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557535a-ac8f-48ba-a476-81b18dfe2e18 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.477833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be86986-aacc-49cc-a636-110db069b822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.478412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be86986-aacc-49cc-a636-110db069b822 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.478679 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be86986-aacc-49cc-a636-110db069b822 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.478947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be86986-aacc-49cc-a636-110db069b822 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.480085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be86986-aacc-49cc-a636-110db069b822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.481683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be86986-aacc-49cc-a636-110db069b822 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.493939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1260/2543] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:45.496959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c42878-5148-49b8-95f3-e1722aa950e8 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.508565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c42878-5148-49b8-95f3-e1722aa950e8 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.514245 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9ff34-c7c7-47f3-9f3c-aba271677dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.515662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9ff34-c7c7-47f3-9f3c-aba271677dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.515951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9ff34-c7c7-47f3-9f3c-aba271677dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.516257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9ff34-c7c7-47f3-9f3c-aba271677dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.516908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9ff34-c7c7-47f3-9f3c-aba271677dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.518770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9ff34-c7c7-47f3-9f3c-aba271677dd7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.530439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1284/2544] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:45.534629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be86986-aacc-49cc-a636-110db069b822 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.545762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be86986-aacc-49cc-a636-110db069b822 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.551559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.552227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.552511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.552801 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] PATH_INFO: `/v3/projects/c984837812674787b3a84e2ba0441f6c/users/af3f55b39be44f9bbd2c4a0344803261/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.574429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c984837812674787b3a84e2ba0441f6c, user_id=af3f55b39be44f9bbd2c4a0344803261, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.577154 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.591391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1261/2545] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:45 2022] PUT /identity/v3/projects/c984837812674787b3a84e2ba0441f6c/users/af3f55b39be44f9bbd2c4a0344803261/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:45.602727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c54d9-c2c8-4894-b331-74f8b4b732f4 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.603271 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c54d9-c2c8-4894-b331-74f8b4b732f4 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.603552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c54d9-c2c8-4894-b331-74f8b4b732f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.643825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-b59c54d9-c2c8-4894-b331-74f8b4b732f4 None None] MFA Rules not processed for user `af3f55b39be44f9bbd2c4a0344803261`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:45.645061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b59c54d9-c2c8-4894-b331-74f8b4b732f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.694977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.696844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.719102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1285/2546] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:45.723626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c54d9-c2c8-4894-b331-74f8b4b732f4 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.765258 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2bd46-d8d8-49cc-8f1c-eb68f0e1d964 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.770288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.770917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.771195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.771473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] PATH_INFO: `/v3/users/9594569f26164f74a99aabb787d9df2c` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.772592 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9594569f26164f74a99aabb787d9df2c)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.774563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.785216 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c54d9-c2c8-4894-b331-74f8b4b732f4 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.789864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e004835-631f-4378-b941-08524131d909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.790459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004835-631f-4378-b941-08524131d909 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.790736 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004835-631f-4378-b941-08524131d909 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.791002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e004835-631f-4378-b941-08524131d909 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.791615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e004835-631f-4378-b941-08524131d909 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.794169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e004835-631f-4378-b941-08524131d909 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.803664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:45.822039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1286/2547] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:45.825779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e004835-631f-4378-b941-08524131d909 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.828188 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9594569f26164f74a99aabb787d9df2c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.836326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9594569f26164f74a99aabb787d9df2c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.837056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9594569f26164f74a99aabb787d9df2c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.852062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9594569f26164f74a99aabb787d9df2c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.860031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e817a1d8-fbbf-4a5d-a337-1608c20d28d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9594569f26164f74a99aabb787d9df2c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:45.869044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1262/2548] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:45 2022] DELETE /identity/v3/users/9594569f26164f74a99aabb787d9df2c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:45.873565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019250-83f9-4627-82b6-7bc0a4658697 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.873885 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019250-83f9-4627-82b6-7bc0a4658697 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.874165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19019250-83f9-4627-82b6-7bc0a4658697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.877803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:45.893440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-19019250-83f9-4627-82b6-7bc0a4658697 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:45.894740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-19019250-83f9-4627-82b6-7bc0a4658697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.898112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e004835-631f-4378-b941-08524131d909 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:45.959380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1263/2549] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:45.961155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:45.962290 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:45.962582 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:45.962893 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:45.963543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:45.968228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:45.970462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19019250-83f9-4627-82b6-7bc0a4658697 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:45.971774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-19019250-83f9-4627-82b6-7bc0a4658697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:45.994770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1287/2550] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:45 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:25:45.999557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:46.001399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:46.014091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc3cc6-e5f9-4c4d-85c1-ed1ca0e54ddd admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:46.020719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d57e8e-5a2a-42b7-b8d3-746e099827b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:46.021336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57e8e-5a2a-42b7-b8d3-746e099827b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.021634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57e8e-5a2a-42b7-b8d3-746e099827b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.021922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d57e8e-5a2a-42b7-b8d3-746e099827b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.022604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d57e8e-5a2a-42b7-b8d3-746e099827b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:46.024238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d57e8e-5a2a-42b7-b8d3-746e099827b1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:46.035322 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1288/2551] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:46.039366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d57e8e-5a2a-42b7-b8d3-746e099827b1 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:46.041562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19019250-83f9-4627-82b6-7bc0a4658697 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:46.046055 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9d2cf-cd11-4f39-b2a3-f21fbc0f14c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:46.046662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9d2cf-cd11-4f39-b2a3-f21fbc0f14c6 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.047058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9d2cf-cd11-4f39-b2a3-f21fbc0f14c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.047336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9d2cf-cd11-4f39-b2a3-f21fbc0f14c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.049143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d57e8e-5a2a-42b7-b8d3-746e099827b1 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:46.054456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c750b9c-5f18-4b38-bcf7-e61af642a65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:46.055093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750b9c-5f18-4b38-bcf7-e61af642a65f admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.055425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750b9c-5f18-4b38-bcf7-e61af642a65f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.055703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c750b9c-5f18-4b38-bcf7-e61af642a65f admin admin] PATH_INFO: `/v3/projects/7da9f6d44e2f4b52b0fafb410f54660b/users/7f5f7046aa9e435b9dcf8ead001263a2/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.056839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9d2cf-cd11-4f39-b2a3-f21fbc0f14c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:46.059433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9d2cf-cd11-4f39-b2a3-f21fbc0f14c6 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:46.069554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1264/2552] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:25:46.071688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c750b9c-5f18-4b38-bcf7-e61af642a65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da9f6d44e2f4b52b0fafb410f54660b, user_id=7f5f7046aa9e435b9dcf8ead001263a2, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:46.075021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c750b9c-5f18-4b38-bcf7-e61af642a65f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:46.087947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1289/2553] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:46 2022] PUT /identity/v3/projects/7da9f6d44e2f4b52b0fafb410f54660b/users/7f5f7046aa9e435b9dcf8ead001263a2/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:46.092500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9d2cf-cd11-4f39-b2a3-f21fbc0f14c6 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:46.102237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9d2cf-cd11-4f39-b2a3-f21fbc0f14c6 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:46.149946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:46.165597 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e39dca-fe10-45a3-bb0d-bc977c77737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:46.166235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39dca-fe10-45a3-bb0d-bc977c77737f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.166598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39dca-fe10-45a3-bb0d-bc977c77737f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.166880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e39dca-fe10-45a3-bb0d-bc977c77737f admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.167565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e39dca-fe10-45a3-bb0d-bc977c77737f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:46.169213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e39dca-fe10-45a3-bb0d-bc977c77737f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:46.179754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1265/2554] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:25:46.183968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c750b9c-5f18-4b38-bcf7-e61af642a65f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:46.194190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c750b9c-5f18-4b38-bcf7-e61af642a65f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:46.199413 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:46.200065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.200340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.200604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] PATH_INFO: `/v3/projects/7da9f6d44e2f4b52b0fafb410f54660b/users/7f5f7046aa9e435b9dcf8ead001263a2/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.202540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da9f6d44e2f4b52b0fafb410f54660b, user_id=7f5f7046aa9e435b9dcf8ead001263a2, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:46.205168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:46.215930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1290/2555] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:46 2022] PUT /identity/v3/projects/7da9f6d44e2f4b52b0fafb410f54660b/users/7f5f7046aa9e435b9dcf8ead001263a2/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:46.226298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52114c-49ff-4e57-a1e7-c89a5dc17c60 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.226639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52114c-49ff-4e57-a1e7-c89a5dc17c60 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.226933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52114c-49ff-4e57-a1e7-c89a5dc17c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.256828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-fd52114c-49ff-4e57-a1e7-c89a5dc17c60 None None] MFA Rules not processed for user `7f5f7046aa9e435b9dcf8ead001263a2`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:46.258217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fd52114c-49ff-4e57-a1e7-c89a5dc17c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:46.341254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1266/2556] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:46.354127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:46.355930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:46.358480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52114c-49ff-4e57-a1e7-c89a5dc17c60 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:46.360448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fd52114c-49ff-4e57-a1e7-c89a5dc17c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:46.435712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52114c-49ff-4e57-a1e7-c89a5dc17c60 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:46.440634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:46.441297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.441629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.441907 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] PATH_INFO: `/v3/projects/48314ec93bb047728923f740a35d7126` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.443105 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48314ec93bb047728923f740a35d7126)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:46.445085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:46.499114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48314ec93bb047728923f740a35d7126', 'request_id': 'req-7d3842ac-8a71-40fb-a29b-c4e32d039e25'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.504909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48314ec93bb047728923f740a35d7126', 'request_id': 'req-7d3842ac-8a71-40fb-a29b-c4e32d039e25'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.512222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48314ec93bb047728923f740a35d7126', 'request_id': 'req-7d3842ac-8a71-40fb-a29b-c4e32d039e25'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.520728 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:46.527308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] The token cache is being invalidate because project 48314ec93bb047728923f740a35d7126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:46.527866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.528626 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.530249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1267/2557] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:46 2022] DELETE /identity/v3/projects/48314ec93bb047728923f740a35d7126 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:46.534153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:46.535715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:46.553124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ba5aa-0c45-425a-8567-1af57734d7ae admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:46.596925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:46.603825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:46.605108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.605568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.605904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.606815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3842ac-8a71-40fb-a29b-c4e32d039e25 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:46.607919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:46.611725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:46.612351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:46.612629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:46.613120 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] PATH_INFO: `/v3/users/1f7ab5a580bd436cb38e8f77f887387e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:46.614261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7ab5a580bd436cb38e8f77f887387e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:46.616190 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:46.663763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7ab5a580bd436cb38e8f77f887387e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.669167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:46.670650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:46.671029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7ab5a580bd436cb38e8f77f887387e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.671717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7ab5a580bd436cb38e8f77f887387e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.680710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1291/2558] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:46.684983 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7ab5a580bd436cb38e8f77f887387e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.692203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-6486c2b9-13f4-4430-96b3-9e31dd565065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7ab5a580bd436cb38e8f77f887387e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:46.698447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1268/2559] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:46 2022] DELETE /identity/v3/users/1f7ab5a580bd436cb38e8f77f887387e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:25:47.015735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.055488 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073259a0-ed24-41db-9df8-a9c83b818bef service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.131154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.131792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.132072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.132341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] PATH_INFO: `/v3/projects/59c28df0248e42ee97eb553dbbb337fd` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.133461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c28df0248e42ee97eb553dbbb337fd)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.135386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.180692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c28df0248e42ee97eb553dbbb337fd', 'request_id': 'req-6d9256ca-9023-4507-b2fb-0580c2347d38'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:47.186164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c28df0248e42ee97eb553dbbb337fd', 'request_id': 'req-6d9256ca-9023-4507-b2fb-0580c2347d38'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:47.192090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c28df0248e42ee97eb553dbbb337fd', 'request_id': 'req-6d9256ca-9023-4507-b2fb-0580c2347d38'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:47.204316 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] The token cache is being invalidate because project 59c28df0248e42ee97eb553dbbb337fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:47.204620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:47.205293 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6d9256ca-9023-4507-b2fb-0580c2347d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:47.206785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1292/2560] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:47 2022] DELETE /identity/v3/projects/59c28df0248e42ee97eb553dbbb337fd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:47.211655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571f66d-1be1-4fd3-b545-2930db1ce9b4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.211965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571f66d-1be1-4fd3-b545-2930db1ce9b4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.212263 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571f66d-1be1-4fd3-b545-2930db1ce9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.232743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e571f66d-1be1-4fd3-b545-2930db1ce9b4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:47.233964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e571f66d-1be1-4fd3-b545-2930db1ce9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:47.295655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1269/2561] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:47.303295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3decd58-1812-49b5-b08e-48bad3c1eb87 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.303578 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3decd58-1812-49b5-b08e-48bad3c1eb87 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.303850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3decd58-1812-49b5-b08e-48bad3c1eb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.323692 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d3decd58-1812-49b5-b08e-48bad3c1eb87 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:47.324912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d3decd58-1812-49b5-b08e-48bad3c1eb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:47.332737 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1293/2562] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:47.336960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571f66d-1be1-4fd3-b545-2930db1ce9b4 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.347342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571f66d-1be1-4fd3-b545-2930db1ce9b4 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.352244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.352839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.353122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.353397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.354080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:25:47.354400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.356537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.368074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1270/2563] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:25:47.372877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3decd58-1812-49b5-b08e-48bad3c1eb87 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.381986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3decd58-1812-49b5-b08e-48bad3c1eb87 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.386597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc74d6-2ae8-4798-9119-af94f2ef8fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.387197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc74d6-2ae8-4798-9119-af94f2ef8fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.387459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc74d6-2ae8-4798-9119-af94f2ef8fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.387731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc74d6-2ae8-4798-9119-af94f2ef8fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.388349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc74d6-2ae8-4798-9119-af94f2ef8fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.390297 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc74d6-2ae8-4798-9119-af94f2ef8fe6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.416866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1294/2564] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:47 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:47.421764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.428100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:47.432100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b51f4f-26c4-4cd6-af23-bd9830d75b2b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.493104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b91d9-22c8-4697-8cd4-981202b33a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.494022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b91d9-22c8-4697-8cd4-981202b33a1a admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.494306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b91d9-22c8-4697-8cd4-981202b33a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.494579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b91d9-22c8-4697-8cd4-981202b33a1a admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.495275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b91d9-22c8-4697-8cd4-981202b33a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.497238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b91d9-22c8-4697-8cd4-981202b33a1a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.519150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1271/2565] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:47 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:47.523255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc74d6-2ae8-4798-9119-af94f2ef8fe6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.530525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:47.534585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc74d6-2ae8-4798-9119-af94f2ef8fe6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.540910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983f1f0-d4ec-400e-89a3-2acec551b98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.541715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983f1f0-d4ec-400e-89a3-2acec551b98c admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.542166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983f1f0-d4ec-400e-89a3-2acec551b98c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.542554 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983f1f0-d4ec-400e-89a3-2acec551b98c admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.543341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983f1f0-d4ec-400e-89a3-2acec551b98c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.544987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983f1f0-d4ec-400e-89a3-2acec551b98c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.555015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1295/2566] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:47.558917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b91d9-22c8-4697-8cd4-981202b33a1a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.568781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b91d9-22c8-4697-8cd4-981202b33a1a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.574176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb76de-b1b6-4522-871e-a61eb1133fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.574782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb76de-b1b6-4522-871e-a61eb1133fee admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.575052 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb76de-b1b6-4522-871e-a61eb1133fee admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.575320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb76de-b1b6-4522-871e-a61eb1133fee admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.575955 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb76de-b1b6-4522-871e-a61eb1133fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.577571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb76de-b1b6-4522-871e-a61eb1133fee admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.588260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1272/2567] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:47.593002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983f1f0-d4ec-400e-89a3-2acec551b98c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.602769 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983f1f0-d4ec-400e-89a3-2acec551b98c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.607390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6134c1b-d012-42a5-96f0-d5078618a764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.608031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6134c1b-d012-42a5-96f0-d5078618a764 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.608309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6134c1b-d012-42a5-96f0-d5078618a764 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.608577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6134c1b-d012-42a5-96f0-d5078618a764 admin admin] PATH_INFO: `/v3/projects/3d9329b6babf4b1e9a51fc4cbe40e9a2/users/82c0496863b44ae18c354c146be10c8a/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.623094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6134c1b-d012-42a5-96f0-d5078618a764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d9329b6babf4b1e9a51fc4cbe40e9a2, user_id=82c0496863b44ae18c354c146be10c8a, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.625854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6134c1b-d012-42a5-96f0-d5078618a764 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.639144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1296/2568] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:47 2022] PUT /identity/v3/projects/3d9329b6babf4b1e9a51fc4cbe40e9a2/users/82c0496863b44ae18c354c146be10c8a/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:47.650331 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2884-413b-41e6-ae4a-fbe0b0886b9a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.650617 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2884-413b-41e6-ae4a-fbe0b0886b9a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.650918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d2884-413b-41e6-ae4a-fbe0b0886b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.678375 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-967d2884-413b-41e6-ae4a-fbe0b0886b9a None None] MFA Rules not processed for user `82c0496863b44ae18c354c146be10c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:47.679574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-967d2884-413b-41e6-ae4a-fbe0b0886b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:47.741891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1273/2569] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:47.746821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6134c1b-d012-42a5-96f0-d5078618a764 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.806810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6134c1b-d012-42a5-96f0-d5078618a764 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.811881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f769-66f1-4472-8cc1-5f46e0d00ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.812474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f769-66f1-4472-8cc1-5f46e0d00ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.812752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f769-66f1-4472-8cc1-5f46e0d00ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.813024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1f769-66f1-4472-8cc1-5f46e0d00ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.813678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1f769-66f1-4472-8cc1-5f46e0d00ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.815559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e1f769-66f1-4472-8cc1-5f46e0d00ad1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.847069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1297/2570] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:25:47 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:47.851257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d2884-413b-41e6-ae4a-fbe0b0886b9a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.862364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d2884-413b-41e6-ae4a-fbe0b0886b9a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.918014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563a219-fc8d-4f05-92a4-7cc9ce44d666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.918625 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563a219-fc8d-4f05-92a4-7cc9ce44d666 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.918909 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563a219-fc8d-4f05-92a4-7cc9ce44d666 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.919181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563a219-fc8d-4f05-92a4-7cc9ce44d666 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.919802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563a219-fc8d-4f05-92a4-7cc9ce44d666 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.921780 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563a219-fc8d-4f05-92a4-7cc9ce44d666 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.942959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1274/2571] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:25:47 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:25:47.946996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f769-66f1-4472-8cc1-5f46e0d00ad1 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.958291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1f769-66f1-4472-8cc1-5f46e0d00ad1 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.962928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40732c3b-78ce-4e2c-89d6-671b83c20f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.963517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40732c3b-78ce-4e2c-89d6-671b83c20f38 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.963816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40732c3b-78ce-4e2c-89d6-671b83c20f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.964086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40732c3b-78ce-4e2c-89d6-671b83c20f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:47.964715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40732c3b-78ce-4e2c-89d6-671b83c20f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:47.966334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40732c3b-78ce-4e2c-89d6-671b83c20f38 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:47.976036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1298/2572] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:47.980004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563a219-fc8d-4f05-92a4-7cc9ce44d666 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:47.985930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:47.991518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563a219-fc8d-4f05-92a4-7cc9ce44d666 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:47.998389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782680ad-c7c5-4272-a6c8-975f38ad9218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:47.999048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782680ad-c7c5-4272-a6c8-975f38ad9218 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:47.999327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782680ad-c7c5-4272-a6c8-975f38ad9218 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:47.999594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782680ad-c7c5-4272-a6c8-975f38ad9218 admin admin] PATH_INFO: `/v3/projects/6b27b5da80324b0ba54121ec41b23e02/users/c41335b08d284f178571132b7645ef25/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:48.021422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782680ad-c7c5-4272-a6c8-975f38ad9218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b27b5da80324b0ba54121ec41b23e02, user_id=c41335b08d284f178571132b7645ef25, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:48.024334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782680ad-c7c5-4272-a6c8-975f38ad9218 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:48.040175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1275/2573] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:47 2022] PUT /identity/v3/projects/6b27b5da80324b0ba54121ec41b23e02/users/c41335b08d284f178571132b7645ef25/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:48.044235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40732c3b-78ce-4e2c-89d6-671b83c20f38 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:48.054440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40732c3b-78ce-4e2c-89d6-671b83c20f38 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:48.113483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c362-561f-4f6c-839e-56ca0dc200c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:48.114170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c362-561f-4f6c-839e-56ca0dc200c5 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:48.114454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c362-561f-4f6c-839e-56ca0dc200c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:48.114728 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c362-561f-4f6c-839e-56ca0dc200c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:48.115372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c362-561f-4f6c-839e-56ca0dc200c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:48.116956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c362-561f-4f6c-839e-56ca0dc200c5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:48.127830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1299/2574] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:25:48.131897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782680ad-c7c5-4272-a6c8-975f38ad9218 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:48.143065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782680ad-c7c5-4272-a6c8-975f38ad9218 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:48.147873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:48.148549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:48.148837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:48.149115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] PATH_INFO: `/v3/projects/6b27b5da80324b0ba54121ec41b23e02/users/c41335b08d284f178571132b7645ef25/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:48.151135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b27b5da80324b0ba54121ec41b23e02, user_id=c41335b08d284f178571132b7645ef25, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:48.154075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:48.165341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1276/2575] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:25:48 2022] PUT /identity/v3/projects/6b27b5da80324b0ba54121ec41b23e02/users/c41335b08d284f178571132b7645ef25/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:48.179348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeaad2a-83a7-4056-b90e-3deb627daf8a None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:48.179644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeaad2a-83a7-4056-b90e-3deb627daf8a None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:48.179914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeaad2a-83a7-4056-b90e-3deb627daf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:48.208551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-0aeaad2a-83a7-4056-b90e-3deb627daf8a None None] MFA Rules not processed for user `c41335b08d284f178571132b7645ef25`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:48.209796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0aeaad2a-83a7-4056-b90e-3deb627daf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:48.239482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:48.282726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1300/2576] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:48.294124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:48.295879 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:48.357932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49b121-81b4-49e6-9f14-d87e7a1f253f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:48.361544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa88bf-e094-44ca-a8ee-7c1cd5ab80b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:48.362272 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa88bf-e094-44ca-a8ee-7c1cd5ab80b3 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:48.362559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa88bf-e094-44ca-a8ee-7c1cd5ab80b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:48.362831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa88bf-e094-44ca-a8ee-7c1cd5ab80b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:48.430530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa88bf-e094-44ca-a8ee-7c1cd5ab80b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:48.432019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fa88bf-e094-44ca-a8ee-7c1cd5ab80b3 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:48.446782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1277/2577] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:50.974597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeaad2a-83a7-4056-b90e-3deb627daf8a None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:50.975968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0aeaad2a-83a7-4056-b90e-3deb627daf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:50.987989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeaad2a-83a7-4056-b90e-3deb627daf8a None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:50.991572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:50.992163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:50.992435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:50.992699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:50.994650 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:51.061160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:51.062609 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:51.071948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1301/2578] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:54.127173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa88bf-e094-44ca-a8ee-7c1cd5ab80b3 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:54.135591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa88bf-e094-44ca-a8ee-7c1cd5ab80b3 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:54.139422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715e62e-c5aa-44eb-9d3f-9497baed67fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:54.140048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715e62e-c5aa-44eb-9d3f-9497baed67fd service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:54.140443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715e62e-c5aa-44eb-9d3f-9497baed67fd service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:54.140725 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715e62e-c5aa-44eb-9d3f-9497baed67fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:54.149575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715e62e-c5aa-44eb-9d3f-9497baed67fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:54.151060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715e62e-c5aa-44eb-9d3f-9497baed67fd service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:54.160793 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1278/2579] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:59.117201 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:59.119005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:59.181550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed9005-1530-4a56-b2bc-3cb91fc2f41c service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:59.186142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:59.186765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:59.187044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:59.187314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] PATH_INFO: `/v3/users/af3f55b39be44f9bbd2c4a0344803261` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:59.188425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3f55b39be44f9bbd2c4a0344803261)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:59.190448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:59.235570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3f55b39be44f9bbd2c4a0344803261'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.242527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3f55b39be44f9bbd2c4a0344803261'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.243160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3f55b39be44f9bbd2c4a0344803261'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.248719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:59.256880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3f55b39be44f9bbd2c4a0344803261'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.264443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3f55b39be44f9bbd2c4a0344803261'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.270469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1302/2580] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:59 2022] DELETE /identity/v3/users/af3f55b39be44f9bbd2c4a0344803261 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:59.274990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58652c6-b4c8-42ea-ab42-8739899f153c None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:59.275462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58652c6-b4c8-42ea-ab42-8739899f153c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:59.275586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58652c6-b4c8-42ea-ab42-8739899f153c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:59.294969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-d58652c6-b4c8-42ea-ab42-8739899f153c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:25:59.296151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d58652c6-b4c8-42ea-ab42-8739899f153c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:59.354493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1279/2581] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:25:59.364829 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:59.366256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:59.425221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c771eff-6fad-4174-ac7c-f94fa28f40f5 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:59.428984 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50121b6e-d9f6-4903-b9ae-8f0ee5b98511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:59.429637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50121b6e-d9f6-4903-b9ae-8f0ee5b98511 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:59.430437 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50121b6e-d9f6-4903-b9ae-8f0ee5b98511 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:59.430717 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50121b6e-d9f6-4903-b9ae-8f0ee5b98511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:59.440187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50121b6e-d9f6-4903-b9ae-8f0ee5b98511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:59.441715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50121b6e-d9f6-4903-b9ae-8f0ee5b98511 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:59.452189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1303/2582] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:59.743532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58652c6-b4c8-42ea-ab42-8739899f153c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:59.745047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d58652c6-b4c8-42ea-ab42-8739899f153c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:59.759732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58652c6-b4c8-42ea-ab42-8739899f153c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:59.766297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:59.767608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:59.768063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:59.768455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] PATH_INFO: `/v3/projects/c984837812674787b3a84e2ba0441f6c` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:59.769501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c984837812674787b3a84e2ba0441f6c)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:59.771795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:59.773978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:59.834942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c984837812674787b3a84e2ba0441f6c', 'request_id': 'req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.848691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c984837812674787b3a84e2ba0441f6c', 'request_id': 'req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.857075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c984837812674787b3a84e2ba0441f6c', 'request_id': 'req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.870712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] The token cache is being invalidate because project c984837812674787b3a84e2ba0441f6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:25:59.871005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.871654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:25:59.873693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1280/2583] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:25:59 2022] DELETE /identity/v3/projects/c984837812674787b3a84e2ba0441f6c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:25:59.877816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50121b6e-d9f6-4903-b9ae-8f0ee5b98511 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:59.889283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50121b6e-d9f6-4903-b9ae-8f0ee5b98511 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:59.948373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:59.949351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:59.949763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:59.950669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] PATH_INFO: `/v3/users/7f5f7046aa9e435b9dcf8ead001263a2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:59.951963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5f7046aa9e435b9dcf8ead001263a2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:59.954009 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:00.003955 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5f7046aa9e435b9dcf8ead001263a2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.015044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5f7046aa9e435b9dcf8ead001263a2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.015721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5f7046aa9e435b9dcf8ead001263a2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.040919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5f7046aa9e435b9dcf8ead001263a2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.054045 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-f90c7702-1fae-4e68-a0ca-5a269ab24586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5f7046aa9e435b9dcf8ead001263a2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.056250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:00.062664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1304/2584] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:25:59 2022] DELETE /identity/v3/users/7f5f7046aa9e435b9dcf8ead001263a2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:00.340106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:00.341747 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:00.354799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed0d6f-a959-4ce2-9295-6ef5a7fb5702 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:00.412438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:00.413010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.413291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.413565 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] PATH_INFO: `/v3/projects/7da9f6d44e2f4b52b0fafb410f54660b` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.414709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7da9f6d44e2f4b52b0fafb410f54660b)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:00.416634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:00.461594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7da9f6d44e2f4b52b0fafb410f54660b', 'request_id': 'req-50434f25-fe64-43f3-8015-c77eef67daf5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.467874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7da9f6d44e2f4b52b0fafb410f54660b', 'request_id': 'req-50434f25-fe64-43f3-8015-c77eef67daf5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.473916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7da9f6d44e2f4b52b0fafb410f54660b', 'request_id': 'req-50434f25-fe64-43f3-8015-c77eef67daf5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.486622 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] The token cache is being invalidate because project 7da9f6d44e2f4b52b0fafb410f54660b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:00.486911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.487535 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-50434f25-fe64-43f3-8015-c77eef67daf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:00.489012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1281/2585] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:00 2022] DELETE /identity/v3/projects/7da9f6d44e2f4b52b0fafb410f54660b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:00.496512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8a14c-d983-4906-bd28-00e1867fd3bf None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.497022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8a14c-d983-4906-bd28-00e1867fd3bf None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.497150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8a14c-d983-4906-bd28-00e1867fd3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.516147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-1bc8a14c-d983-4906-bd28-00e1867fd3bf None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:00.517332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1bc8a14c-d983-4906-bd28-00e1867fd3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:00.574248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1305/2586] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:00.582681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e795-af72-4927-9bbe-2bb69514f408 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.583044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e795-af72-4927-9bbe-2bb69514f408 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.583247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217e795-af72-4927-9bbe-2bb69514f408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.603009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e217e795-af72-4927-9bbe-2bb69514f408 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:00.604204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e217e795-af72-4927-9bbe-2bb69514f408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:00.612593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1282/2587] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:00.616616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8a14c-d983-4906-bd28-00e1867fd3bf None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:00.626600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8a14c-d983-4906-bd28-00e1867fd3bf None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:00.631227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:00.631819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.632095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.632366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.633029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:00.633312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:00.634941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:00.646210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1306/2588] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:00.650339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217e795-af72-4927-9bbe-2bb69514f408 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:00.659951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e217e795-af72-4927-9bbe-2bb69514f408 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:00.664603 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3097-c8ac-498c-84d5-016164c6131a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:00.665191 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3097-c8ac-498c-84d5-016164c6131a admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.665469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3097-c8ac-498c-84d5-016164c6131a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.665774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b3097-c8ac-498c-84d5-016164c6131a admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.666401 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b3097-c8ac-498c-84d5-016164c6131a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:00.668326 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b3097-c8ac-498c-84d5-016164c6131a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:00.679770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:00.697033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1283/2589] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:00 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:00.701189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:00.711375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98145ae-d1e6-4913-bbc5-00b44cd6bc31 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:00.773035 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4023-f5af-422b-9f7c-88d1aa6b7783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:00.773723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4023-f5af-422b-9f7c-88d1aa6b7783 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.773997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4023-f5af-422b-9f7c-88d1aa6b7783 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.774270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a4023-f5af-422b-9f7c-88d1aa6b7783 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.774901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a4023-f5af-422b-9f7c-88d1aa6b7783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:00.776788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a4023-f5af-422b-9f7c-88d1aa6b7783 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:00.804126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1307/2590] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:00 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:00.808113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3097-c8ac-498c-84d5-016164c6131a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:00.818378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b3097-c8ac-498c-84d5-016164c6131a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:00.822996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314f1c9-04e9-46b7-8cf1-9fe74e2d1b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:00.823587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314f1c9-04e9-46b7-8cf1-9fe74e2d1b04 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.823878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314f1c9-04e9-46b7-8cf1-9fe74e2d1b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.824144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314f1c9-04e9-46b7-8cf1-9fe74e2d1b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.824782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314f1c9-04e9-46b7-8cf1-9fe74e2d1b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:00.826476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314f1c9-04e9-46b7-8cf1-9fe74e2d1b04 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:00.836225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1284/2591] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:00.842088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4023-f5af-422b-9f7c-88d1aa6b7783 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:00.851844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a4023-f5af-422b-9f7c-88d1aa6b7783 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:00.858004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6037a9a5-c1e8-41ad-9a5b-570182fc548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:00.858600 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a9a5-c1e8-41ad-9a5b-570182fc548b admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.858880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a9a5-c1e8-41ad-9a5b-570182fc548b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.859155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037a9a5-c1e8-41ad-9a5b-570182fc548b admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.859798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037a9a5-c1e8-41ad-9a5b-570182fc548b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:00.861403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6037a9a5-c1e8-41ad-9a5b-570182fc548b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:00.872063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1308/2592] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:00.876224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314f1c9-04e9-46b7-8cf1-9fe74e2d1b04 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:00.886220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314f1c9-04e9-46b7-8cf1-9fe74e2d1b04 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:00.891407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:00.892083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.892369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.892649 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] PATH_INFO: `/v3/projects/e061663dc9694f178fe835fcb0707aab/users/55dbb379b7c84005a05c3678f84ae87e/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:00.907449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e061663dc9694f178fe835fcb0707aab, user_id=55dbb379b7c84005a05c3678f84ae87e, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:00.910170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:00.922402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1285/2593] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:00 2022] PUT /identity/v3/projects/e061663dc9694f178fe835fcb0707aab/users/55dbb379b7c84005a05c3678f84ae87e/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:00.988508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631572f-589c-4f0a-807c-5b4b540b9996 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:00.988797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631572f-589c-4f0a-807c-5b4b540b9996 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:00.989075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631572f-589c-4f0a-807c-5b4b540b9996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:01.017204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-2631572f-589c-4f0a-807c-5b4b540b9996 None None] MFA Rules not processed for user `55dbb379b7c84005a05c3678f84ae87e`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:01.018454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2631572f-589c-4f0a-807c-5b4b540b9996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:01.080726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:01.091701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1309/2594] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:01.102542 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:01.104227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:01.160415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740f231-f74a-40bc-97aa-e808974469d4 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:01.165371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e6ce-c3cd-4a19-9fc5-7b389c7aac3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:01.166006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e6ce-c3cd-4a19-9fc5-7b389c7aac3e service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:01.166288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e6ce-c3cd-4a19-9fc5-7b389c7aac3e service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:01.166564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35e6ce-c3cd-4a19-9fc5-7b389c7aac3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:01.174833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35e6ce-c3cd-4a19-9fc5-7b389c7aac3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:01.176261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35e6ce-c3cd-4a19-9fc5-7b389c7aac3e service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:01.185102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1286/2595] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:11.088180 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:11.181072 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:15.155475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631572f-589c-4f0a-807c-5b4b540b9996 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:15.157822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2631572f-589c-4f0a-807c-5b4b540b9996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:15.226223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631572f-589c-4f0a-807c-5b4b540b9996 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:15.231765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b737e-8012-41d5-9efe-632e4ed53f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:15.232460 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b737e-8012-41d5-9efe-632e4ed53f9f service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:15.233115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b737e-8012-41d5-9efe-632e4ed53f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:15.233386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b737e-8012-41d5-9efe-632e4ed53f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:15.235224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a10b737e-8012-41d5-9efe-632e4ed53f9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:15.307704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b737e-8012-41d5-9efe-632e4ed53f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:15.309059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b737e-8012-41d5-9efe-632e4ed53f9f service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:15.319990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1310/2596] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:24.660241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e6ce-c3cd-4a19-9fc5-7b389c7aac3e service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:24.723114 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35e6ce-c3cd-4a19-9fc5-7b389c7aac3e service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:24.727809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:24.728515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.728803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.729079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] PATH_INFO: `/v3/users/55dbb379b7c84005a05c3678f84ae87e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.730224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55dbb379b7c84005a05c3678f84ae87e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:24.732513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:24.784104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55dbb379b7c84005a05c3678f84ae87e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.791305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55dbb379b7c84005a05c3678f84ae87e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.791947 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55dbb379b7c84005a05c3678f84ae87e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.803831 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55dbb379b7c84005a05c3678f84ae87e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.810708 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55dbb379b7c84005a05c3678f84ae87e'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.816490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1287/2597] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:24 2022] DELETE /identity/v3/users/55dbb379b7c84005a05c3678f84ae87e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:24.820946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e68ac9-b385-4eb6-84a1-98e290570b48 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.821423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e68ac9-b385-4eb6-84a1-98e290570b48 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.821530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e68ac9-b385-4eb6-84a1-98e290570b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.840779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-34e68ac9-b385-4eb6-84a1-98e290570b48 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:24.842017 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-34e68ac9-b385-4eb6-84a1-98e290570b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:24.900031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1311/2598] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:24.910435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:24.911824 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:24.970355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b786e3d5-939f-44bd-8297-d4f92d508917 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:24.974201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:24.974800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.975080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.975352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.984301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:24.985763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:24.995244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1288/2599] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:25.252899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e68ac9-b385-4eb6-84a1-98e290570b48 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.254434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-34e68ac9-b385-4eb6-84a1-98e290570b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.267562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e68ac9-b385-4eb6-84a1-98e290570b48 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.272213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.272844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.273124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.273397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] PATH_INFO: `/v3/projects/e061663dc9694f178fe835fcb0707aab` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.274552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e061663dc9694f178fe835fcb0707aab)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.276461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.315956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:25.325917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e061663dc9694f178fe835fcb0707aab', 'request_id': 'req-ec88aec4-9a40-408f-ad99-0e5ca21fc968'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.332039 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e061663dc9694f178fe835fcb0707aab', 'request_id': 'req-ec88aec4-9a40-408f-ad99-0e5ca21fc968'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.338368 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e061663dc9694f178fe835fcb0707aab', 'request_id': 'req-ec88aec4-9a40-408f-ad99-0e5ca21fc968'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.350353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] The token cache is being invalidate because project e061663dc9694f178fe835fcb0707aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:25.350639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.351251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ec88aec4-9a40-408f-ad99-0e5ca21fc968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.353114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1312/2600] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:25 2022] DELETE /identity/v3/projects/e061663dc9694f178fe835fcb0707aab => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:25.410828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.412899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.426208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42742b93-1517-44b0-b2c8-335ae3282867 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.446426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:25.489816 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.490596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.490954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.491301 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] PATH_INFO: `/v3/users/82c0496863b44ae18c354c146be10c8a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.492547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c0496863b44ae18c354c146be10c8a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.494876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.549852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c0496863b44ae18c354c146be10c8a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.557951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c0496863b44ae18c354c146be10c8a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.558657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c0496863b44ae18c354c146be10c8a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.572875 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c0496863b44ae18c354c146be10c8a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.580109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c0496863b44ae18c354c146be10c8a'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.590800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1289/2601] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:25 2022] DELETE /identity/v3/users/82c0496863b44ae18c354c146be10c8a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:25.593042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee02a4-e199-4ace-8a96-1e9ee844acde None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.593240 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee02a4-e199-4ace-8a96-1e9ee844acde None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.593484 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee02a4-e199-4ace-8a96-1e9ee844acde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.615269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-aaee02a4-e199-4ace-8a96-1e9ee844acde None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:25.616491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-aaee02a4-e199-4ace-8a96-1e9ee844acde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.684780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1313/2602] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:25.695455 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.696867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.764181 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7fa840-33b7-4024-bf78-4bdc7e38ac24 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.767160 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d02437-4ef6-4207-8637-c31f4d56b6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.767766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02437-4ef6-4207-8637-c31f4d56b6de service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.768049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02437-4ef6-4207-8637-c31f4d56b6de service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.768324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d02437-4ef6-4207-8637-c31f4d56b6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.777549 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d02437-4ef6-4207-8637-c31f4d56b6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.779002 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d02437-4ef6-4207-8637-c31f4d56b6de service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.789204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1290/2603] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:26.068440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee02a4-e199-4ace-8a96-1e9ee844acde None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.070071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-aaee02a4-e199-4ace-8a96-1e9ee844acde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.082974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee02a4-e199-4ace-8a96-1e9ee844acde None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.087572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.088195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.088471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.088824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] PATH_INFO: `/v3/projects/3d9329b6babf4b1e9a51fc4cbe40e9a2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.090032 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d9329b6babf4b1e9a51fc4cbe40e9a2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.092343 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.140247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d9329b6babf4b1e9a51fc4cbe40e9a2', 'request_id': 'req-a68c53bb-0863-488f-9367-67497b050db4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.146038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d9329b6babf4b1e9a51fc4cbe40e9a2', 'request_id': 'req-a68c53bb-0863-488f-9367-67497b050db4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.152448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d9329b6babf4b1e9a51fc4cbe40e9a2', 'request_id': 'req-a68c53bb-0863-488f-9367-67497b050db4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.166549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] The token cache is being invalidate because project 3d9329b6babf4b1e9a51fc4cbe40e9a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:26.167023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.167843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.169687 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1314/2604] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:26 2022] DELETE /identity/v3/projects/3d9329b6babf4b1e9a51fc4cbe40e9a2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:26.174462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d02437-4ef6-4207-8637-c31f4d56b6de service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.186351 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d02437-4ef6-4207-8637-c31f4d56b6de service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.227089 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:26.244083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.244709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.245005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.245260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] PATH_INFO: `/v3/users/c41335b08d284f178571132b7645ef25` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.246405 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41335b08d284f178571132b7645ef25)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.248320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.293149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41335b08d284f178571132b7645ef25'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.300575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41335b08d284f178571132b7645ef25'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.301182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41335b08d284f178571132b7645ef25'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.312852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41335b08d284f178571132b7645ef25'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.319528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a634cb77-3d20-4b86-bc7c-7fd535366608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41335b08d284f178571132b7645ef25'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.325229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1291/2605] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:26 2022] DELETE /identity/v3/users/c41335b08d284f178571132b7645ef25 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:26.591130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.592754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.606496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68c53bb-0863-488f-9367-67497b050db4 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.652078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:26.665669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.666309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.666589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.666863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] PATH_INFO: `/v3/projects/6b27b5da80324b0ba54121ec41b23e02` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.667977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b27b5da80324b0ba54121ec41b23e02)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.669973 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.715672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b27b5da80324b0ba54121ec41b23e02', 'request_id': 'req-b561b2ed-b1e4-4050-a742-30f28683d780'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.721055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b27b5da80324b0ba54121ec41b23e02', 'request_id': 'req-b561b2ed-b1e4-4050-a742-30f28683d780'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.727044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b27b5da80324b0ba54121ec41b23e02', 'request_id': 'req-b561b2ed-b1e4-4050-a742-30f28683d780'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.740269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] The token cache is being invalidate because project 6b27b5da80324b0ba54121ec41b23e02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:26.740561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.741190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b561b2ed-b1e4-4050-a742-30f28683d780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:26.742947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1315/2606] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:26 2022] DELETE /identity/v3/projects/6b27b5da80324b0ba54121ec41b23e02 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:26.747894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29388c-fd07-4f54-a6af-c61114cf478d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.748593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29388c-fd07-4f54-a6af-c61114cf478d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.748593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29388c-fd07-4f54-a6af-c61114cf478d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.769523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-5a29388c-fd07-4f54-a6af-c61114cf478d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:26.770772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5a29388c-fd07-4f54-a6af-c61114cf478d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.828579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1292/2607] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:26.837350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9a612-b92a-41a6-9057-3fb8aa3a5033 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.837661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9a612-b92a-41a6-9057-3fb8aa3a5033 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.837940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9a612-b92a-41a6-9057-3fb8aa3a5033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.857355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-60a9a612-b92a-41a6-9057-3fb8aa3a5033 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:26.858646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-60a9a612-b92a-41a6-9057-3fb8aa3a5033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.866756 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1316/2608] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:26.871046 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29388c-fd07-4f54-a6af-c61114cf478d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.881218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29388c-fd07-4f54-a6af-c61114cf478d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.886060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.886714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.887001 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.887281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.887959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:26.888354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.890732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.906302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1293/2609] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:26.907332 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9a612-b92a-41a6-9057-3fb8aa3a5033 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.916630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9a612-b92a-41a6-9057-3fb8aa3a5033 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.925179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.925825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.926101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.926465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.927747 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.929745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.968492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1317/2610] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/projects => generated 391 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:26.977402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.981936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.983604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.985035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd85b9d8-7eab-43d9-8fb3-8728d3598890 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.036850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:27.048546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47238c3-df08-422a-88a0-bc1f5b83f33d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.048936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919d461-13ed-4f02-b57e-82b381f9f2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.049551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919d461-13ed-4f02-b57e-82b381f9f2fa admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.049861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919d461-13ed-4f02-b57e-82b381f9f2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.050164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919d461-13ed-4f02-b57e-82b381f9f2fa admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.050825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919d461-13ed-4f02-b57e-82b381f9f2fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.052373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.052830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4919d461-13ed-4f02-b57e-82b381f9f2fa admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.052975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.053257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.053533 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.055325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:27.069372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.070934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.075558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1294/2611] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:27.079645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919d461-13ed-4f02-b57e-82b381f9f2fa admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.081218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1318/2612] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:26:27.089245 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919d461-13ed-4f02-b57e-82b381f9f2fa admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.094015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae93090-d702-463f-8f1d-af8f6894359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.094726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93090-d702-463f-8f1d-af8f6894359c admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.095014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93090-d702-463f-8f1d-af8f6894359c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.095290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93090-d702-463f-8f1d-af8f6894359c admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.096021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae93090-d702-463f-8f1d-af8f6894359c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.097700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae93090-d702-463f-8f1d-af8f6894359c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.109013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1295/2613] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:27.113080 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.123163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839faa56-86ec-4c22-b871-c2414c28faf0 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.128411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17839b69-2be6-47de-83e5-8f1579979e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.129041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839b69-2be6-47de-83e5-8f1579979e8b admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.129776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839b69-2be6-47de-83e5-8f1579979e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.129776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839b69-2be6-47de-83e5-8f1579979e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.130251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17839b69-2be6-47de-83e5-8f1579979e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.131849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17839b69-2be6-47de-83e5-8f1579979e8b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.142002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1319/2614] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:27.146184 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae93090-d702-463f-8f1d-af8f6894359c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.156548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae93090-d702-463f-8f1d-af8f6894359c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.160864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b20954-e110-49eb-a26d-3e459c4fe03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.161518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b20954-e110-49eb-a26d-3e459c4fe03e admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.161846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b20954-e110-49eb-a26d-3e459c4fe03e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.162636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b20954-e110-49eb-a26d-3e459c4fe03e admin admin] PATH_INFO: `/v3/projects/b957bd04ad95455b92873059795af1d5/users/0080213ac1574e90bd55fa70f6b8e9f9/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.179598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b20954-e110-49eb-a26d-3e459c4fe03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b957bd04ad95455b92873059795af1d5, user_id=0080213ac1574e90bd55fa70f6b8e9f9, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.182368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b20954-e110-49eb-a26d-3e459c4fe03e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.198225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1296/2615] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:27 2022] PUT /identity/v3/projects/b957bd04ad95455b92873059795af1d5/users/0080213ac1574e90bd55fa70f6b8e9f9/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:27.208429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa587432-2f09-45ea-9b91-e368a3eca1d1 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.208803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa587432-2f09-45ea-9b91-e368a3eca1d1 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.209319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa587432-2f09-45ea-9b91-e368a3eca1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.246115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fa587432-2f09-45ea-9b91-e368a3eca1d1 None None] MFA Rules not processed for user `0080213ac1574e90bd55fa70f6b8e9f9`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:27.247538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fa587432-2f09-45ea-9b91-e368a3eca1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:27.279013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:27.316491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1320/2616] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:27.321025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b20954-e110-49eb-a26d-3e459c4fe03e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.385720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b20954-e110-49eb-a26d-3e459c4fe03e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.390426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac9f3d-e4e4-40f4-aa55-9d0f87378110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.391034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac9f3d-e4e4-40f4-aa55-9d0f87378110 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.391319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac9f3d-e4e4-40f4-aa55-9d0f87378110 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.391595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac9f3d-e4e4-40f4-aa55-9d0f87378110 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.392243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac9f3d-e4e4-40f4-aa55-9d0f87378110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.394506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac9f3d-e4e4-40f4-aa55-9d0f87378110 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.425678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1297/2617] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:27 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:27.427088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa587432-2f09-45ea-9b91-e368a3eca1d1 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.438567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa587432-2f09-45ea-9b91-e368a3eca1d1 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.497045 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270838e-17fd-4139-83b1-0f4a2e09c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.497666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270838e-17fd-4139-83b1-0f4a2e09c9bf admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.498013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270838e-17fd-4139-83b1-0f4a2e09c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.498287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270838e-17fd-4139-83b1-0f4a2e09c9bf admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.498985 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270838e-17fd-4139-83b1-0f4a2e09c9bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.500921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270838e-17fd-4139-83b1-0f4a2e09c9bf admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.521928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1321/2618] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:27 2022] POST /identity/v3/users => generated 303 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:27.525830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac9f3d-e4e4-40f4-aa55-9d0f87378110 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.536529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac9f3d-e4e4-40f4-aa55-9d0f87378110 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.541526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee910e-a8ed-4033-975e-1ea49e79ef8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.542165 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee910e-a8ed-4033-975e-1ea49e79ef8f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.542453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee910e-a8ed-4033-975e-1ea49e79ef8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.542730 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ee910e-a8ed-4033-975e-1ea49e79ef8f admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.543415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee910e-a8ed-4033-975e-1ea49e79ef8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.545040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ee910e-a8ed-4033-975e-1ea49e79ef8f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.557944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1298/2619] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:27.562158 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270838e-17fd-4139-83b1-0f4a2e09c9bf admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.572465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270838e-17fd-4139-83b1-0f4a2e09c9bf admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.577778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06011fb1-6c2b-40f6-bc67-5a3fffadf241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.578444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06011fb1-6c2b-40f6-bc67-5a3fffadf241 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.578896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06011fb1-6c2b-40f6-bc67-5a3fffadf241 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.579174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06011fb1-6c2b-40f6-bc67-5a3fffadf241 admin admin] PATH_INFO: `/v3/projects/ba2360276d784ebfa0519eb664e0f1b9/users/c6ad7d012af240d699af6b54c9e64033/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.594346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06011fb1-6c2b-40f6-bc67-5a3fffadf241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2360276d784ebfa0519eb664e0f1b9, user_id=c6ad7d012af240d699af6b54c9e64033, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.596997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06011fb1-6c2b-40f6-bc67-5a3fffadf241 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.610994 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1322/2620] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:27 2022] PUT /identity/v3/projects/ba2360276d784ebfa0519eb664e0f1b9/users/c6ad7d012af240d699af6b54c9e64033/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:27.615029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee910e-a8ed-4033-975e-1ea49e79ef8f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.625268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ee910e-a8ed-4033-975e-1ea49e79ef8f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.681201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aee4e-21f9-4429-b8a1-09584b216482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.681868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aee4e-21f9-4429-b8a1-09584b216482 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.682152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aee4e-21f9-4429-b8a1-09584b216482 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.682427 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aee4e-21f9-4429-b8a1-09584b216482 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.683068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aee4e-21f9-4429-b8a1-09584b216482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.684663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aee4e-21f9-4429-b8a1-09584b216482 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.695000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1299/2621] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:27.699378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06011fb1-6c2b-40f6-bc67-5a3fffadf241 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.709873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06011fb1-6c2b-40f6-bc67-5a3fffadf241 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.714680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.715334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.715629 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.715897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] PATH_INFO: `/v3/projects/ba2360276d784ebfa0519eb664e0f1b9/users/c6ad7d012af240d699af6b54c9e64033/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.718386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2360276d784ebfa0519eb664e0f1b9, user_id=c6ad7d012af240d699af6b54c9e64033, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.721018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.731988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1323/2622] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:27 2022] PUT /identity/v3/projects/ba2360276d784ebfa0519eb664e0f1b9/users/c6ad7d012af240d699af6b54c9e64033/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:27.743390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa47332-a291-4df5-9b4a-6c5d8c8d5f86 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.743763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa47332-a291-4df5-9b4a-6c5d8c8d5f86 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.743941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa47332-a291-4df5-9b4a-6c5d8c8d5f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.772825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3aa47332-a291-4df5-9b4a-6c5d8c8d5f86 None None] MFA Rules not processed for user `c6ad7d012af240d699af6b54c9e64033`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:27.773987 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3aa47332-a291-4df5-9b4a-6c5d8c8d5f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:27.778119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:27.843566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1300/2623] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:27.855912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.857654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:27.918585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737443f0-0aed-4f9c-9393-61b4397cae35 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.922423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.923046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.923314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.923572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.934138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.935765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.946516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1324/2624] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:27 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:28.004274 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa47332-a291-4df5-9b4a-6c5d8c8d5f86 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:28.005799 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3aa47332-a291-4df5-9b4a-6c5d8c8d5f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:28.031449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa47332-a291-4df5-9b4a-6c5d8c8d5f86 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:28.035682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63eb96-4a22-4e92-bc8d-4761c29833bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:28.036367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63eb96-4a22-4e92-bc8d-4761c29833bb service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:28.036720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63eb96-4a22-4e92-bc8d-4761c29833bb service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:28.037060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63eb96-4a22-4e92-bc8d-4761c29833bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:28.098365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63eb96-4a22-4e92-bc8d-4761c29833bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:28.100063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63eb96-4a22-4e92-bc8d-4761c29833bb service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:28.110768 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1301/2625] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:30.055685 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.057182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:30.117166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:30.158742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b42315-ba5f-4239-9d37-e84bdf0e498e service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:30.163251 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:30.163883 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.164159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.164432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] PATH_INFO: `/v3/users/61b9ae7499154e5eab4f3c23d8bb6785` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.165545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b9ae7499154e5eab4f3c23d8bb6785)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:30.167560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:30.221314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b9ae7499154e5eab4f3c23d8bb6785'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.229500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b9ae7499154e5eab4f3c23d8bb6785'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.230313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b9ae7499154e5eab4f3c23d8bb6785'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.243517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b9ae7499154e5eab4f3c23d8bb6785'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.251041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d490cb05-ce26-471d-b7bd-72f43c9b76e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b9ae7499154e5eab4f3c23d8bb6785'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.257399 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1325/2626] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:30 2022] DELETE /identity/v3/users/61b9ae7499154e5eab4f3c23d8bb6785 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:30.448472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63eb96-4a22-4e92-bc8d-4761c29833bb service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.468575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63eb96-4a22-4e92-bc8d-4761c29833bb service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:30.527576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:30.528244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.528528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.528803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] PATH_INFO: `/v3/projects/552017cc73234b8db2ac15a8496e649e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.530486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=552017cc73234b8db2ac15a8496e649e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:30.532414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:30.573533 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:30.580566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552017cc73234b8db2ac15a8496e649e', 'request_id': 'req-ac0e6321-a0e4-4f05-bed2-425bd3352f60'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.587763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552017cc73234b8db2ac15a8496e649e', 'request_id': 'req-ac0e6321-a0e4-4f05-bed2-425bd3352f60'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.594605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552017cc73234b8db2ac15a8496e649e', 'request_id': 'req-ac0e6321-a0e4-4f05-bed2-425bd3352f60'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.606867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] The token cache is being invalidate because project 552017cc73234b8db2ac15a8496e649e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:30.607159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.607786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ac0e6321-a0e4-4f05-bed2-425bd3352f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.609892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1302/2627] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:30 2022] DELETE /identity/v3/projects/552017cc73234b8db2ac15a8496e649e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:30.614989 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac318-b0da-4442-b750-d5f91f3f7a9f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.615270 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac318-b0da-4442-b750-d5f91f3f7a9f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.615541 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cac318-b0da-4442-b750-d5f91f3f7a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.634810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-f2cac318-b0da-4442-b750-d5f91f3f7a9f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:30.635990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-f2cac318-b0da-4442-b750-d5f91f3f7a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:30.692438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1326/2628] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:30.700532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065bcd5-3668-42d0-b350-64aa74242ad9 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.700822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065bcd5-3668-42d0-b350-64aa74242ad9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.701100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065bcd5-3668-42d0-b350-64aa74242ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.721077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-2065bcd5-3668-42d0-b350-64aa74242ad9 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:30.722255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2065bcd5-3668-42d0-b350-64aa74242ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:30.730474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1303/2629] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:30.735797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cac318-b0da-4442-b750-d5f91f3f7a9f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.746778 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cac318-b0da-4442-b750-d5f91f3f7a9f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:30.751645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:30.752255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.752531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.752880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.753429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:30.753870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:30.755483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:30.768972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1327/2630] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:30.774049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065bcd5-3668-42d0-b350-64aa74242ad9 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.784500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065bcd5-3668-42d0-b350-64aa74242ad9 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:30.789991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc661-96b7-41a0-bd3c-90aaf80eebb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:30.790739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc661-96b7-41a0-bd3c-90aaf80eebb7 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.791135 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc661-96b7-41a0-bd3c-90aaf80eebb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.791505 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc661-96b7-41a0-bd3c-90aaf80eebb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.792238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcc661-96b7-41a0-bd3c-90aaf80eebb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:30.794358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcc661-96b7-41a0-bd3c-90aaf80eebb7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:30.834880 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1304/2631] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:30 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:30.839050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.850572 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b130e79-f854-4f00-83cb-af7648e6c40c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:30.884287 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:30.916435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a31a01-16c9-4fa6-ae1e-36e544e8ac0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:30.917063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31a01-16c9-4fa6-ae1e-36e544e8ac0f admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.917343 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31a01-16c9-4fa6-ae1e-36e544e8ac0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.917624 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a31a01-16c9-4fa6-ae1e-36e544e8ac0f admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.918349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a31a01-16c9-4fa6-ae1e-36e544e8ac0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:30.920359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a31a01-16c9-4fa6-ae1e-36e544e8ac0f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:30.943749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1328/2632] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:30 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:30.947923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc661-96b7-41a0-bd3c-90aaf80eebb7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.959246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc661-96b7-41a0-bd3c-90aaf80eebb7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:30.965720 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857190f-27e1-4044-b18a-6bb269041d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:30.966449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857190f-27e1-4044-b18a-6bb269041d60 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.966791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857190f-27e1-4044-b18a-6bb269041d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.967164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857190f-27e1-4044-b18a-6bb269041d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.968451 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857190f-27e1-4044-b18a-6bb269041d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:30.970159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857190f-27e1-4044-b18a-6bb269041d60 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:30.984661 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1305/2633] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:30.988429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a31a01-16c9-4fa6-ae1e-36e544e8ac0f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.999862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a31a01-16c9-4fa6-ae1e-36e544e8ac0f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:31.005121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511a61f-49d9-48d7-89e2-ccdb1817062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:31.005826 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a61f-49d9-48d7-89e2-ccdb1817062c admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.006107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a61f-49d9-48d7-89e2-ccdb1817062c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.006379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a61f-49d9-48d7-89e2-ccdb1817062c admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.007024 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511a61f-49d9-48d7-89e2-ccdb1817062c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:31.008623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511a61f-49d9-48d7-89e2-ccdb1817062c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:31.023519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1329/2634] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:31.027696 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857190f-27e1-4044-b18a-6bb269041d60 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:31.038496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857190f-27e1-4044-b18a-6bb269041d60 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:31.043115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:31.043754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.044033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.044307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] PATH_INFO: `/v3/projects/969a788d047540c4aef1463b351a35f3/users/ff3c58b6de124399a5216cd150fb4de0/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.058511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=969a788d047540c4aef1463b351a35f3, user_id=ff3c58b6de124399a5216cd150fb4de0, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:31.061170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:31.073878 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1306/2635] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:31 2022] PUT /identity/v3/projects/969a788d047540c4aef1463b351a35f3/users/ff3c58b6de124399a5216cd150fb4de0/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:31.079236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9157d-bcd5-4733-997f-ff197c2991bb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.079774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9157d-bcd5-4733-997f-ff197c2991bb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.079774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9157d-bcd5-4733-997f-ff197c2991bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.098886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-0fa9157d-bcd5-4733-997f-ff197c2991bb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:31.100069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0fa9157d-bcd5-4733-997f-ff197c2991bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:31.157695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1330/2636] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:31.167259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:31.168949 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:31.224707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1784d75d-0a6c-45a0-9588-6d5c00696cd4 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:31.228362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4efcb-508e-4a24-9aef-8f70abf66878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:31.228960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4efcb-508e-4a24-9aef-8f70abf66878 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.229241 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4efcb-508e-4a24-9aef-8f70abf66878 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.229517 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4efcb-508e-4a24-9aef-8f70abf66878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.238416 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4efcb-508e-4a24-9aef-8f70abf66878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:31.239810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4efcb-508e-4a24-9aef-8f70abf66878 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:31.249939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1307/2637] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:31.611587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9157d-bcd5-4733-997f-ff197c2991bb None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:31.613115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0fa9157d-bcd5-4733-997f-ff197c2991bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:31.626007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9157d-bcd5-4733-997f-ff197c2991bb None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:31.631090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:31.631880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.632850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.632850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] PATH_INFO: `/v3/users/0080213ac1574e90bd55fa70f6b8e9f9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.633819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0080213ac1574e90bd55fa70f6b8e9f9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:31.636208 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:31.685353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0080213ac1574e90bd55fa70f6b8e9f9'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.694041 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0080213ac1574e90bd55fa70f6b8e9f9'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.694762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0080213ac1574e90bd55fa70f6b8e9f9'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.707776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0080213ac1574e90bd55fa70f6b8e9f9'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.714943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0080213ac1574e90bd55fa70f6b8e9f9'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.721342 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1331/2638] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:31 2022] DELETE /identity/v3/users/0080213ac1574e90bd55fa70f6b8e9f9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:31.726123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070492a-e1b9-4af4-965b-6d369cf720f8 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.726679 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070492a-e1b9-4af4-965b-6d369cf720f8 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.726894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070492a-e1b9-4af4-965b-6d369cf720f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.749790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3070492a-e1b9-4af4-965b-6d369cf720f8 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:31.751036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3070492a-e1b9-4af4-965b-6d369cf720f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:31.813595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1308/2639] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:31.823295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:31.825411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:31.855547 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:31.891429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91a629-caad-4f0a-8e43-e306fb3ba452 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:31.895934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:31.896549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.896863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.897186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.908164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72415d-c712-4df2-a89f-0fc10852b381 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.908164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72415d-c712-4df2-a89f-0fc10852b381 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.908164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72415d-c712-4df2-a89f-0fc10852b381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.912815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:31.914570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:31.927749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1332/2640] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:31.934700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:31.950059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-4e72415d-c712-4df2-a89f-0fc10852b381 None None] MFA Rules not processed for user `ff3c58b6de124399a5216cd150fb4de0`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:31.951662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-4e72415d-c712-4df2-a89f-0fc10852b381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.040446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1309/2641] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:32.045154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.047543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.061317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fcf33-a209-401e-aeec-eef789b67ab7 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.066898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42236b6-351a-4bce-87d3-d020c989b325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.067617 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42236b6-351a-4bce-87d3-d020c989b325 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.067965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42236b6-351a-4bce-87d3-d020c989b325 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.068333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42236b6-351a-4bce-87d3-d020c989b325 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.069077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42236b6-351a-4bce-87d3-d020c989b325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.071104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42236b6-351a-4bce-87d3-d020c989b325 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.117051 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1333/2642] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:32 2022] POST /identity/v3/projects => generated 403 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:32.121555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72415d-c712-4df2-a89f-0fc10852b381 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.132868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72415d-c712-4df2-a89f-0fc10852b381 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.194770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.195397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.195686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.195953 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.196590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.198608 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.229712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1310/2643] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:32 2022] POST /identity/v3/users => generated 308 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:32.233803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42236b6-351a-4bce-87d3-d020c989b325 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.256200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.258095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.258789 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42236b6-351a-4bce-87d3-d020c989b325 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.265177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8d018-1023-4e39-9396-066e2ed4e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.265857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8d018-1023-4e39-9396-066e2ed4e8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.266123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8d018-1023-4e39-9396-066e2ed4e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.266398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e8d018-1023-4e39-9396-066e2ed4e8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.267489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8d018-1023-4e39-9396-066e2ed4e8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.269242 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e8d018-1023-4e39-9396-066e2ed4e8d2 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.272930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1e0a5-84d8-442e-891d-f3d940264892 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.278715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.280527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.280891 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.281552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1334/2644] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:32.284633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] PATH_INFO: `/v3/projects/b957bd04ad95455b92873059795af1d5` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.285984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b957bd04ad95455b92873059795af1d5)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.286331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8d018-1023-4e39-9396-066e2ed4e8d2 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.288088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.297449 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e8d018-1023-4e39-9396-066e2ed4e8d2 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.302970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbde8e-2b52-49e9-86b1-efa341b63a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.303859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbde8e-2b52-49e9-86b1-efa341b63a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.304237 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbde8e-2b52-49e9-86b1-efa341b63a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.304510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbde8e-2b52-49e9-86b1-efa341b63a87 admin admin] PATH_INFO: `/v3/projects/9f1976b2a44c44d0bad0fb42ca77ed89/users/30087196ad454a9b92ef75c65588c7a4/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.320285 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbde8e-2b52-49e9-86b1-efa341b63a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1976b2a44c44d0bad0fb42ca77ed89, user_id=30087196ad454a9b92ef75c65588c7a4, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.323104 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbde8e-2b52-49e9-86b1-efa341b63a87 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.337479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1335/2645] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:32 2022] PUT /identity/v3/projects/9f1976b2a44c44d0bad0fb42ca77ed89/users/30087196ad454a9b92ef75c65588c7a4/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:32.337722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b957bd04ad95455b92873059795af1d5', 'request_id': 'req-582f3966-24ff-4523-9cbb-f56367e6d1e4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.341339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbde8e-2b52-49e9-86b1-efa341b63a87 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.344012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b957bd04ad95455b92873059795af1d5', 'request_id': 'req-582f3966-24ff-4523-9cbb-f56367e6d1e4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.351125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b957bd04ad95455b92873059795af1d5', 'request_id': 'req-582f3966-24ff-4523-9cbb-f56367e6d1e4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.364215 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] The token cache is being invalidate because project b957bd04ad95455b92873059795af1d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:32.364497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.365136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.366702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1311/2646] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:32 2022] DELETE /identity/v3/projects/b957bd04ad95455b92873059795af1d5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:32.370892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.372270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.386829 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:32.411150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbde8e-2b52-49e9-86b1-efa341b63a87 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.415981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.416608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.416910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.417187 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.417851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.419432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.429639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1336/2647] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:32.433576 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.435239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.442914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582f3966-24ff-4523-9cbb-f56367e6d1e4 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.447374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106d58c-e7b4-4fff-a3b6-8e393cca1daa admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.447514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.448125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.448403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.448672 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] PATH_INFO: `/v3/users/c6ad7d012af240d699af6b54c9e64033` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.449814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6ad7d012af240d699af6b54c9e64033)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.452261 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.452540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.453174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.453454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.453923 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] PATH_INFO: `/v3/projects/9f1976b2a44c44d0bad0fb42ca77ed89/users/30087196ad454a9b92ef75c65588c7a4/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.455850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1976b2a44c44d0bad0fb42ca77ed89, user_id=30087196ad454a9b92ef75c65588c7a4, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.458509 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.469136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1337/2648] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:32 2022] PUT /identity/v3/projects/9f1976b2a44c44d0bad0fb42ca77ed89/users/30087196ad454a9b92ef75c65588c7a4/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:32.501530 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6ad7d012af240d699af6b54c9e64033'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.509234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6ad7d012af240d699af6b54c9e64033'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.509915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6ad7d012af240d699af6b54c9e64033'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.524113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6ad7d012af240d699af6b54c9e64033'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.531832 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-9908c95d-f7cf-407c-9f12-b5e0fef2f728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6ad7d012af240d699af6b54c9e64033'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.538334 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1312/2649] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:32 2022] DELETE /identity/v3/users/c6ad7d012af240d699af6b54c9e64033 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:32.837073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.839102 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.910177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:32.924165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7396cf-1af0-450a-9d1b-ae86cdcf3153 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.982807 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.983458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.983764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.984042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] PATH_INFO: `/v3/projects/ba2360276d784ebfa0519eb664e0f1b9` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.985181 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2360276d784ebfa0519eb664e0f1b9)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.987130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.038142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2360276d784ebfa0519eb664e0f1b9', 'request_id': 'req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.043966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2360276d784ebfa0519eb664e0f1b9', 'request_id': 'req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.050660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2360276d784ebfa0519eb664e0f1b9', 'request_id': 'req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.064903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] The token cache is being invalidate because project ba2360276d784ebfa0519eb664e0f1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:33.065261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.065912 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c6f5d48f-fae4-4f89-b41d-3470eb9ce7ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.067478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1338/2650] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:32 2022] DELETE /identity/v3/projects/ba2360276d784ebfa0519eb664e0f1b9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:33.072469 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d46a62-8499-461b-9bd7-9b371b76ba42 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.072991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d46a62-8499-461b-9bd7-9b371b76ba42 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.073107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d46a62-8499-461b-9bd7-9b371b76ba42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.093740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-a5d46a62-8499-461b-9bd7-9b371b76ba42 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:33.094976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a5d46a62-8499-461b-9bd7-9b371b76ba42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:33.149466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df806-7b8a-4d7b-9d2d-b76a1c45a45b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.150065 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df806-7b8a-4d7b-9d2d-b76a1c45a45b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.150207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df806-7b8a-4d7b-9d2d-b76a1c45a45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.178042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1313/2651] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:33.179579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-b52df806-7b8a-4d7b-9d2d-b76a1c45a45b None None] MFA Rules not processed for user `30087196ad454a9b92ef75c65588c7a4`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:33.180901 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b52df806-7b8a-4d7b-9d2d-b76a1c45a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:33.246091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c056-ccfa-4572-bae6-c2cd568fd6eb None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.246412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c056-ccfa-4572-bae6-c2cd568fd6eb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.246693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3c056-ccfa-4572-bae6-c2cd568fd6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.267265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1339/2652] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:33.268193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-91b3c056-ccfa-4572-bae6-c2cd568fd6eb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:33.272054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-91b3c056-ccfa-4572-bae6-c2cd568fd6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:33.285740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1314/2653] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:33.290140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52df806-7b8a-4d7b-9d2d-b76a1c45a45b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.302151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52df806-7b8a-4d7b-9d2d-b76a1c45a45b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.307248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.307852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.308210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.308487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.309132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:33.309430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.311670 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.325561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1340/2654] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:33.329796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3c056-ccfa-4572-bae6-c2cd568fd6eb None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.339701 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3c056-ccfa-4572-bae6-c2cd568fd6eb None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.346200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b6878-1123-43cf-be9b-c01becfbbe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.347501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b6878-1123-43cf-be9b-c01becfbbe25 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.347859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b6878-1123-43cf-be9b-c01becfbbe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.348201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b6878-1123-43cf-be9b-c01becfbbe25 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.349400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b6878-1123-43cf-be9b-c01becfbbe25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.351967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b6878-1123-43cf-be9b-c01becfbbe25 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.385686 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1315/2655] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:33.387202 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.392525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:33.397767 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7217a-a0f8-4238-ba8b-6381fca98436 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.458856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c165e7a-2689-49b8-b452-f16201ed06e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.459464 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c165e7a-2689-49b8-b452-f16201ed06e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.459738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c165e7a-2689-49b8-b452-f16201ed06e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.460001 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c165e7a-2689-49b8-b452-f16201ed06e7 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.460623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c165e7a-2689-49b8-b452-f16201ed06e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.462620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c165e7a-2689-49b8-b452-f16201ed06e7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.485278 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1341/2656] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:33.489424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b6878-1123-43cf-be9b-c01becfbbe25 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.495862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:33.502202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b6878-1123-43cf-be9b-c01becfbbe25 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.512497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaff600-9405-49d5-973a-52fbe838673a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.513156 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaff600-9405-49d5-973a-52fbe838673a admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.514103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaff600-9405-49d5-973a-52fbe838673a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.514418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaff600-9405-49d5-973a-52fbe838673a admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.516136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaff600-9405-49d5-973a-52fbe838673a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.517959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaff600-9405-49d5-973a-52fbe838673a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.528476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1316/2657] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:33.533112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c165e7a-2689-49b8-b452-f16201ed06e7 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.543345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c165e7a-2689-49b8-b452-f16201ed06e7 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.548797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a836308-2904-4847-9917-00e56fc3c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.549419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a836308-2904-4847-9917-00e56fc3c7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.549775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a836308-2904-4847-9917-00e56fc3c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.550424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a836308-2904-4847-9917-00e56fc3c7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.550620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a836308-2904-4847-9917-00e56fc3c7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.552199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a836308-2904-4847-9917-00e56fc3c7c3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.563462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1342/2658] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:33.567665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaff600-9405-49d5-973a-52fbe838673a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.577582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaff600-9405-49d5-973a-52fbe838673a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.582449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c5ba6-6615-4daa-b472-dfbdb7701caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.582811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5ba6-6615-4daa-b472-dfbdb7701caa admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.583091 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5ba6-6615-4daa-b472-dfbdb7701caa admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.583363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c5ba6-6615-4daa-b472-dfbdb7701caa admin admin] PATH_INFO: `/v3/projects/263221efc086432f8fe633c48fe1edce/users/61f22da11f064341b299403e2e757d3f/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.600111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c5ba6-6615-4daa-b472-dfbdb7701caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=263221efc086432f8fe633c48fe1edce, user_id=61f22da11f064341b299403e2e757d3f, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.602857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c5ba6-6615-4daa-b472-dfbdb7701caa admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.618320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1317/2659] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:33 2022] PUT /identity/v3/projects/263221efc086432f8fe633c48fe1edce/users/61f22da11f064341b299403e2e757d3f/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:33.628167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba0ba-89ab-403a-a6cd-7bea96d5c65b None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.628566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba0ba-89ab-403a-a6cd-7bea96d5c65b None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.629087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ba0ba-89ab-403a-a6cd-7bea96d5c65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.660091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-0e6ba0ba-89ab-403a-a6cd-7bea96d5c65b None None] MFA Rules not processed for user `61f22da11f064341b299403e2e757d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:33.661425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0e6ba0ba-89ab-403a-a6cd-7bea96d5c65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:33.728857 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1343/2660] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:33.733956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c5ba6-6615-4daa-b472-dfbdb7701caa admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.801298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c5ba6-6615-4daa-b472-dfbdb7701caa admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.809601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b23d01-a7a2-4420-9bea-dc86a7e51394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.810324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b23d01-a7a2-4420-9bea-dc86a7e51394 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.810678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b23d01-a7a2-4420-9bea-dc86a7e51394 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.811107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b23d01-a7a2-4420-9bea-dc86a7e51394 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.811802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b23d01-a7a2-4420-9bea-dc86a7e51394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.813864 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b23d01-a7a2-4420-9bea-dc86a7e51394 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.843341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1318/2661] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:33.847999 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ba0ba-89ab-403a-a6cd-7bea96d5c65b None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.860920 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ba0ba-89ab-403a-a6cd-7bea96d5c65b None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.918678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3baf2-8502-46b5-b312-76da60463a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.919376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3baf2-8502-46b5-b312-76da60463a4b admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.919661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3baf2-8502-46b5-b312-76da60463a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.919932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3baf2-8502-46b5-b312-76da60463a4b admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.920688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3baf2-8502-46b5-b312-76da60463a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.922684 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3baf2-8502-46b5-b312-76da60463a4b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.941852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:33.947300 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1344/2662] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:33.951276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b23d01-a7a2-4420-9bea-dc86a7e51394 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.963009 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b23d01-a7a2-4420-9bea-dc86a7e51394 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.968558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdd957-2785-4666-8dfb-2fe1b441221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.969168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdd957-2785-4666-8dfb-2fe1b441221e admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.969579 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdd957-2785-4666-8dfb-2fe1b441221e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.969889 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bdd957-2785-4666-8dfb-2fe1b441221e admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.970531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdd957-2785-4666-8dfb-2fe1b441221e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.972178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bdd957-2785-4666-8dfb-2fe1b441221e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.984223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1319/2663] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:33.988148 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3baf2-8502-46b5-b312-76da60463a4b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.999582 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3baf2-8502-46b5-b312-76da60463a4b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.004863 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23bec9-a038-411d-a294-27156001f5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.005527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23bec9-a038-411d-a294-27156001f5e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.005905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23bec9-a038-411d-a294-27156001f5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.006174 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23bec9-a038-411d-a294-27156001f5e2 admin admin] PATH_INFO: `/v3/projects/fb03693c39a342879a1f7670bfa7e034/users/d803b53eba9647b0b03d52c786852d5c/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.024977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23bec9-a038-411d-a294-27156001f5e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb03693c39a342879a1f7670bfa7e034, user_id=d803b53eba9647b0b03d52c786852d5c, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.027678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23bec9-a038-411d-a294-27156001f5e2 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.042744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1345/2664] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:33 2022] PUT /identity/v3/projects/fb03693c39a342879a1f7670bfa7e034/users/d803b53eba9647b0b03d52c786852d5c/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:34.047863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdd957-2785-4666-8dfb-2fe1b441221e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.061026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bdd957-2785-4666-8dfb-2fe1b441221e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.121230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4c36b-306a-4626-8b2b-37ae383e2726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.122006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c36b-306a-4626-8b2b-37ae383e2726 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.122391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c36b-306a-4626-8b2b-37ae383e2726 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.122766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c36b-306a-4626-8b2b-37ae383e2726 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.123550 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4c36b-306a-4626-8b2b-37ae383e2726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.125265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4c36b-306a-4626-8b2b-37ae383e2726 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.135745 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1320/2665] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:34.140430 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23bec9-a038-411d-a294-27156001f5e2 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.150668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23bec9-a038-411d-a294-27156001f5e2 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.155865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.156524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.156802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.157077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] PATH_INFO: `/v3/projects/fb03693c39a342879a1f7670bfa7e034/users/d803b53eba9647b0b03d52c786852d5c/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.159057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb03693c39a342879a1f7670bfa7e034, user_id=d803b53eba9647b0b03d52c786852d5c, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.161762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.172258 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1346/2666] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:34 2022] PUT /identity/v3/projects/fb03693c39a342879a1f7670bfa7e034/users/d803b53eba9647b0b03d52c786852d5c/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:34.182484 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8ec16-43a8-42b7-8a4d-54ca83cb827b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.182842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8ec16-43a8-42b7-8a4d-54ca83cb827b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.183119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8ec16-43a8-42b7-8a4d-54ca83cb827b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.213665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-58a8ec16-43a8-42b7-8a4d-54ca83cb827b None None] MFA Rules not processed for user `d803b53eba9647b0b03d52c786852d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:34.215621 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-58a8ec16-43a8-42b7-8a4d-54ca83cb827b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.241998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:34.293063 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1321/2667] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:34.304186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.305974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.342288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8ec16-43a8-42b7-8a4d-54ca83cb827b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.343976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-58a8ec16-43a8-42b7-8a4d-54ca83cb827b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.377502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3163a-c6f0-4104-8be4-8088de22dba7 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.381545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68ef5-06ef-4b2a-aaf6-8b8bf7574f8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.382207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68ef5-06ef-4b2a-aaf6-8b8bf7574f8f service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.382924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68ef5-06ef-4b2a-aaf6-8b8bf7574f8f service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.383231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f68ef5-06ef-4b2a-aaf6-8b8bf7574f8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.427221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8ec16-43a8-42b7-8a4d-54ca83cb827b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.432439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.433074 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.433358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.433657 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] PATH_INFO: `/v3/users/ff3c58b6de124399a5216cd150fb4de0` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.434790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3c58b6de124399a5216cd150fb4de0)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.437307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.450362 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68ef5-06ef-4b2a-aaf6-8b8bf7574f8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.452641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f68ef5-06ef-4b2a-aaf6-8b8bf7574f8f service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.462690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1347/2668] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:34.491344 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3c58b6de124399a5216cd150fb4de0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.501308 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3c58b6de124399a5216cd150fb4de0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.502357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3c58b6de124399a5216cd150fb4de0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.519664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3c58b6de124399a5216cd150fb4de0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.526555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3c58b6de124399a5216cd150fb4de0'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.532666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1322/2669] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:34 2022] DELETE /identity/v3/users/ff3c58b6de124399a5216cd150fb4de0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:34.708199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68ef5-06ef-4b2a-aaf6-8b8bf7574f8f service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.720688 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f68ef5-06ef-4b2a-aaf6-8b8bf7574f8f service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.800907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.801548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.801856 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.802233 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] PATH_INFO: `/v3/projects/969a788d047540c4aef1463b351a35f3` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.803265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=969a788d047540c4aef1463b351a35f3)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.805152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.826889 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:34.853519 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '969a788d047540c4aef1463b351a35f3', 'request_id': 'req-77444e8d-73fd-462e-970f-396578a5f606'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.859878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '969a788d047540c4aef1463b351a35f3', 'request_id': 'req-77444e8d-73fd-462e-970f-396578a5f606'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.866666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '969a788d047540c4aef1463b351a35f3', 'request_id': 'req-77444e8d-73fd-462e-970f-396578a5f606'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.879900 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] The token cache is being invalidate because project 969a788d047540c4aef1463b351a35f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:34.880192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.880821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.882328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1348/2670] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:34 2022] DELETE /identity/v3/projects/969a788d047540c4aef1463b351a35f3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:34.886491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.887950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.900581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebba81a-219f-4072-b9da-6884a842f63e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.939976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:34.957346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.958005 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.958310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.958582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] PATH_INFO: `/v3/users/30087196ad454a9b92ef75c65588c7a4` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.959687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30087196ad454a9b92ef75c65588c7a4)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.961618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.007807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30087196ad454a9b92ef75c65588c7a4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.015516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30087196ad454a9b92ef75c65588c7a4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.016282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30087196ad454a9b92ef75c65588c7a4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.029400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30087196ad454a9b92ef75c65588c7a4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.036788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1477e48e-0e75-4ed5-9b82-8c44919114d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30087196ad454a9b92ef75c65588c7a4'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.042826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1323/2671] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:34 2022] DELETE /identity/v3/users/30087196ad454a9b92ef75c65588c7a4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:35.212913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.214647 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:35.229673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77444e8d-73fd-462e-970f-396578a5f606 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.295109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.295774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.296048 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.296319 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] PATH_INFO: `/v3/projects/9f1976b2a44c44d0bad0fb42ca77ed89` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.297433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1976b2a44c44d0bad0fb42ca77ed89)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.299366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.344739 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1976b2a44c44d0bad0fb42ca77ed89', 'request_id': 'req-508e0dd4-c9bd-46e0-a649-178e852a3bea'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.350405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1976b2a44c44d0bad0fb42ca77ed89', 'request_id': 'req-508e0dd4-c9bd-46e0-a649-178e852a3bea'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.356493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1976b2a44c44d0bad0fb42ca77ed89', 'request_id': 'req-508e0dd4-c9bd-46e0-a649-178e852a3bea'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.368496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] The token cache is being invalidate because project 9f1976b2a44c44d0bad0fb42ca77ed89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:35.368781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.369395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-508e0dd4-c9bd-46e0-a649-178e852a3bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.370887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1349/2672] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:35 2022] DELETE /identity/v3/projects/9f1976b2a44c44d0bad0fb42ca77ed89 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:35.375812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311aa7d-e76e-410b-94fd-5eb408f59d97 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.376315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311aa7d-e76e-410b-94fd-5eb408f59d97 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.376421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311aa7d-e76e-410b-94fd-5eb408f59d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.395999 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-9311aa7d-e76e-410b-94fd-5eb408f59d97 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:35.397179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-9311aa7d-e76e-410b-94fd-5eb408f59d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:35.454570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1324/2673] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:35.462266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9c05-51bc-409a-b929-afbfaf1eccc6 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.462555 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9c05-51bc-409a-b929-afbfaf1eccc6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.462832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc9c05-51bc-409a-b929-afbfaf1eccc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.482873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-92dc9c05-51bc-409a-b929-afbfaf1eccc6 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:35.484072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-92dc9c05-51bc-409a-b929-afbfaf1eccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:35.491874 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1350/2674] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:35.496217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311aa7d-e76e-410b-94fd-5eb408f59d97 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.505744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311aa7d-e76e-410b-94fd-5eb408f59d97 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.510203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.510788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.511064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.511333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.511959 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:35.512259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.513873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.525442 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1325/2675] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:35.529477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc9c05-51bc-409a-b929-afbfaf1eccc6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.539131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc9c05-51bc-409a-b929-afbfaf1eccc6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.543879 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffe772-d889-459d-bf97-111580d00657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.544471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe772-d889-459d-bf97-111580d00657 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.544752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe772-d889-459d-bf97-111580d00657 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.545021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efffe772-d889-459d-bf97-111580d00657 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.545698 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffe772-d889-459d-bf97-111580d00657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.547589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efffe772-d889-459d-bf97-111580d00657 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.577703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1351/2676] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:35 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:35.579711 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.589585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a47d-5ef3-419b-aef1-fec66d0e6f70 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.636029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:35.650067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af719c-eb6a-4d00-8e42-95e3883b74a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.650669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af719c-eb6a-4d00-8e42-95e3883b74a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.650974 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af719c-eb6a-4d00-8e42-95e3883b74a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.651246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af719c-eb6a-4d00-8e42-95e3883b74a8 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.651911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af719c-eb6a-4d00-8e42-95e3883b74a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.653899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af719c-eb6a-4d00-8e42-95e3883b74a8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.675865 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1326/2677] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:35 2022] POST /identity/v3/users => generated 282 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:35.679938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffe772-d889-459d-bf97-111580d00657 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.690587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efffe772-d889-459d-bf97-111580d00657 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.695191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867886-e80f-40cf-a2ef-836b010b7026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.695779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867886-e80f-40cf-a2ef-836b010b7026 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.696060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867886-e80f-40cf-a2ef-836b010b7026 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.696334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867886-e80f-40cf-a2ef-836b010b7026 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.696975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78867886-e80f-40cf-a2ef-836b010b7026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.698585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78867886-e80f-40cf-a2ef-836b010b7026 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.700604 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:35.710429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1352/2678] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:35.713963 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af719c-eb6a-4d00-8e42-95e3883b74a8 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.723746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af719c-eb6a-4d00-8e42-95e3883b74a8 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.728385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11328b5-eae5-4a6d-8df2-f169a298712f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.728975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11328b5-eae5-4a6d-8df2-f169a298712f admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.729257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11328b5-eae5-4a6d-8df2-f169a298712f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.729531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11328b5-eae5-4a6d-8df2-f169a298712f admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.730207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11328b5-eae5-4a6d-8df2-f169a298712f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.731803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11328b5-eae5-4a6d-8df2-f169a298712f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.865445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1327/2679] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:35.875975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867886-e80f-40cf-a2ef-836b010b7026 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.880764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867886-e80f-40cf-a2ef-836b010b7026 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.885551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84302593-ec40-429d-bb28-f96bbe0d5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.886231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84302593-ec40-429d-bb28-f96bbe0d5c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.886506 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84302593-ec40-429d-bb28-f96bbe0d5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.886774 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84302593-ec40-429d-bb28-f96bbe0d5c80 admin admin] PATH_INFO: `/v3/projects/a889dd814b5f4862ad337877cab6db1b/users/baeaefeabbc64616843bfb7a50cd2db7/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.904063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84302593-ec40-429d-bb28-f96bbe0d5c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a889dd814b5f4862ad337877cab6db1b, user_id=baeaefeabbc64616843bfb7a50cd2db7, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.906904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84302593-ec40-429d-bb28-f96bbe0d5c80 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.922655 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1353/2680] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:35 2022] PUT /identity/v3/projects/a889dd814b5f4862ad337877cab6db1b/users/baeaefeabbc64616843bfb7a50cd2db7/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:35.933779 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d23cc-c9a9-4f7e-b6ff-4310a91f640c None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.934096 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d23cc-c9a9-4f7e-b6ff-4310a91f640c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.934372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d23cc-c9a9-4f7e-b6ff-4310a91f640c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.964556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b00d23cc-c9a9-4f7e-b6ff-4310a91f640c None None] MFA Rules not processed for user `baeaefeabbc64616843bfb7a50cd2db7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:35.965777 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b00d23cc-c9a9-4f7e-b6ff-4310a91f640c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.040813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1328/2681] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:36.045967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84302593-ec40-429d-bb28-f96bbe0d5c80 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.105982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84302593-ec40-429d-bb28-f96bbe0d5c80 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.111226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3cd1db-0d84-4831-b6d1-be107ff8539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.111825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cd1db-0d84-4831-b6d1-be107ff8539e admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.112121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cd1db-0d84-4831-b6d1-be107ff8539e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.112461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cd1db-0d84-4831-b6d1-be107ff8539e admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.113073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3cd1db-0d84-4831-b6d1-be107ff8539e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.115000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3cd1db-0d84-4831-b6d1-be107ff8539e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.143183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1354/2682] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:36 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:36.147252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d23cc-c9a9-4f7e-b6ff-4310a91f640c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.158589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d23cc-c9a9-4f7e-b6ff-4310a91f640c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.220651 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a12014-47ad-4f6f-accb-2e8c807a4fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.221270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a12014-47ad-4f6f-accb-2e8c807a4fbe admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.221551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a12014-47ad-4f6f-accb-2e8c807a4fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.221858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a12014-47ad-4f6f-accb-2e8c807a4fbe admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.222537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a12014-47ad-4f6f-accb-2e8c807a4fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.224486 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a12014-47ad-4f6f-accb-2e8c807a4fbe admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.247297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1329/2683] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:36 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:36.252074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3cd1db-0d84-4831-b6d1-be107ff8539e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.263002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3cd1db-0d84-4831-b6d1-be107ff8539e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.268591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ee4d8-11e1-4b8d-89b2-8458e9afde17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.269176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee4d8-11e1-4b8d-89b2-8458e9afde17 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.269448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee4d8-11e1-4b8d-89b2-8458e9afde17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.269765 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ee4d8-11e1-4b8d-89b2-8458e9afde17 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.270405 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ee4d8-11e1-4b8d-89b2-8458e9afde17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.272014 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ee4d8-11e1-4b8d-89b2-8458e9afde17 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.283345 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1355/2684] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:36.287572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a12014-47ad-4f6f-accb-2e8c807a4fbe admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.297828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a12014-47ad-4f6f-accb-2e8c807a4fbe admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.302523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420ae41-fe1b-41fc-9f23-8a0898f1ce99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.303163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420ae41-fe1b-41fc-9f23-8a0898f1ce99 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.303443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420ae41-fe1b-41fc-9f23-8a0898f1ce99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.303740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420ae41-fe1b-41fc-9f23-8a0898f1ce99 admin admin] PATH_INFO: `/v3/projects/2c63615298c1491bbc5ffd17ade8a489/users/c29010b2219941b9922ef36e1338be40/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.319270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420ae41-fe1b-41fc-9f23-8a0898f1ce99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c63615298c1491bbc5ffd17ade8a489, user_id=c29010b2219941b9922ef36e1338be40, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.321982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420ae41-fe1b-41fc-9f23-8a0898f1ce99 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.334172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1330/2685] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:36 2022] PUT /identity/v3/projects/2c63615298c1491bbc5ffd17ade8a489/users/c29010b2219941b9922ef36e1338be40/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:36.338163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ee4d8-11e1-4b8d-89b2-8458e9afde17 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.347854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ee4d8-11e1-4b8d-89b2-8458e9afde17 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.404540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a07ede-8349-43de-8230-16dfb59933af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.405644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a07ede-8349-43de-8230-16dfb59933af admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.405952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a07ede-8349-43de-8230-16dfb59933af admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.406219 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a07ede-8349-43de-8230-16dfb59933af admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.406849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a07ede-8349-43de-8230-16dfb59933af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.408429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a07ede-8349-43de-8230-16dfb59933af admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.417875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1356/2686] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:36.422783 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420ae41-fe1b-41fc-9f23-8a0898f1ce99 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.432806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420ae41-fe1b-41fc-9f23-8a0898f1ce99 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.437396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.438070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.438348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.438618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] PATH_INFO: `/v3/projects/2c63615298c1491bbc5ffd17ade8a489/users/c29010b2219941b9922ef36e1338be40/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.440518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c63615298c1491bbc5ffd17ade8a489, user_id=c29010b2219941b9922ef36e1338be40, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.443199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.453485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1331/2687] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:36 2022] PUT /identity/v3/projects/2c63615298c1491bbc5ffd17ade8a489/users/c29010b2219941b9922ef36e1338be40/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:36.464601 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9993f5b-b2dd-48d3-84dd-ed3808ac4608 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.464962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9993f5b-b2dd-48d3-84dd-ed3808ac4608 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.465239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9993f5b-b2dd-48d3-84dd-ed3808ac4608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.493959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-c9993f5b-b2dd-48d3-84dd-ed3808ac4608 None None] MFA Rules not processed for user `c29010b2219941b9922ef36e1338be40`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:36.495182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c9993f5b-b2dd-48d3-84dd-ed3808ac4608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.526376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:36.562715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1357/2688] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:36.574178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.575862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.588601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:36.639162 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ff300-5e62-4350-b2f4-fe4c52859eef admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.642869 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.643463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.643741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.644012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.652892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.654342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.663922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1332/2689] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:36.738077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9993f5b-b2dd-48d3-84dd-ed3808ac4608 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.748771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9993f5b-b2dd-48d3-84dd-ed3808ac4608 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.752456 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b89a6-1303-4672-ad25-3c7744b7dbf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.753045 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b89a6-1303-4672-ad25-3c7744b7dbf2 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.753361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b89a6-1303-4672-ad25-3c7744b7dbf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.753654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b89a6-1303-4672-ad25-3c7744b7dbf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.808758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b89a6-1303-4672-ad25-3c7744b7dbf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.810224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b89a6-1303-4672-ad25-3c7744b7dbf2 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.819101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1358/2690] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:37.074121 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.075521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:37.138244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb6b65-fbd1-4a98-927c-6e318e95eec2 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.143291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.143905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.144189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.144462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] PATH_INFO: `/v3/users/61f22da11f064341b299403e2e757d3f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.145570 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f22da11f064341b299403e2e757d3f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.147521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.191899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f22da11f064341b299403e2e757d3f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.198735 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f22da11f064341b299403e2e757d3f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.199406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f22da11f064341b299403e2e757d3f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.211380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f22da11f064341b299403e2e757d3f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.218611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f22da11f064341b299403e2e757d3f'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.224446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1333/2691] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:37 2022] DELETE /identity/v3/users/61f22da11f064341b299403e2e757d3f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:37.229930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5f13-25bd-4ba3-ac2b-10e555047e06 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.230239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5f13-25bd-4ba3-ac2b-10e555047e06 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.230538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5f13-25bd-4ba3-ac2b-10e555047e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.250608 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d07a5f13-25bd-4ba3-ac2b-10e555047e06 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:37.251790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d07a5f13-25bd-4ba3-ac2b-10e555047e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:37.309998 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1359/2692] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:37.319901 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.321282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:37.354828 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:37.381704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced72a9-7c83-4aa1-9066-d816c3101262 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.385539 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fff7ac-2a28-483b-bc46-dc47096982dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.386236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fff7ac-2a28-483b-bc46-dc47096982dd service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.386513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fff7ac-2a28-483b-bc46-dc47096982dd service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.386784 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fff7ac-2a28-483b-bc46-dc47096982dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.396966 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fff7ac-2a28-483b-bc46-dc47096982dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.398429 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fff7ac-2a28-483b-bc46-dc47096982dd service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.413464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1334/2693] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:37.677244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a5f13-25bd-4ba3-ac2b-10e555047e06 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.678695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d07a5f13-25bd-4ba3-ac2b-10e555047e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:37.691573 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a5f13-25bd-4ba3-ac2b-10e555047e06 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.696644 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.697390 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.697708 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.697983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] PATH_INFO: `/v3/projects/263221efc086432f8fe633c48fe1edce` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.699115 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=263221efc086432f8fe633c48fe1edce)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.701090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.751374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '263221efc086432f8fe633c48fe1edce', 'request_id': 'req-64299745-10c9-4b5f-a718-3747fef4d2a4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.756695 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '263221efc086432f8fe633c48fe1edce', 'request_id': 'req-64299745-10c9-4b5f-a718-3747fef4d2a4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.762958 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '263221efc086432f8fe633c48fe1edce', 'request_id': 'req-64299745-10c9-4b5f-a718-3747fef4d2a4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.765175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:37.776470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] The token cache is being invalidate because project 263221efc086432f8fe633c48fe1edce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:37.776758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.777376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.778839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1360/2694] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:37 2022] DELETE /identity/v3/projects/263221efc086432f8fe633c48fe1edce => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:37.783449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fff7ac-2a28-483b-bc46-dc47096982dd service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.793079 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fff7ac-2a28-483b-bc46-dc47096982dd service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.850138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.850765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.851045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.851316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] PATH_INFO: `/v3/users/d803b53eba9647b0b03d52c786852d5c` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.852518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d803b53eba9647b0b03d52c786852d5c)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.854576 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.902106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd803b53eba9647b0b03d52c786852d5c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.910435 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd803b53eba9647b0b03d52c786852d5c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.911109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd803b53eba9647b0b03d52c786852d5c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.925939 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd803b53eba9647b0b03d52c786852d5c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.932915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1bc40537-15d7-44a9-80bc-214f2f322df4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd803b53eba9647b0b03d52c786852d5c'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:37.938809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1335/2695] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:37 2022] DELETE /identity/v3/users/d803b53eba9647b0b03d52c786852d5c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:38.249189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.250818 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:38.303434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64299745-10c9-4b5f-a718-3747fef4d2a4 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.370536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.371363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.371669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.371949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] PATH_INFO: `/v3/projects/fb03693c39a342879a1f7670bfa7e034` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.373072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb03693c39a342879a1f7670bfa7e034)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.375045 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.422200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb03693c39a342879a1f7670bfa7e034', 'request_id': 'req-4fa57eb9-5a10-4ffc-8567-701498518ddd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.428508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb03693c39a342879a1f7670bfa7e034', 'request_id': 'req-4fa57eb9-5a10-4ffc-8567-701498518ddd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.439612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb03693c39a342879a1f7670bfa7e034', 'request_id': 'req-4fa57eb9-5a10-4ffc-8567-701498518ddd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.453830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] The token cache is being invalidate because project fb03693c39a342879a1f7670bfa7e034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:38.454145 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.454981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4fa57eb9-5a10-4ffc-8567-701498518ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.456453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1361/2696] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:38 2022] DELETE /identity/v3/projects/fb03693c39a342879a1f7670bfa7e034 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:38.461340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd677f-5a31-4d8b-9d76-e801ab375b00 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.461647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd677f-5a31-4d8b-9d76-e801ab375b00 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.461933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd677f-5a31-4d8b-9d76-e801ab375b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.481590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-60cd677f-5a31-4d8b-9d76-e801ab375b00 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:38.482796 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-60cd677f-5a31-4d8b-9d76-e801ab375b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:38.528911 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:38.543472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1336/2697] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:38.551865 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d71c7-bc7f-41e3-8f4a-ab9c65e20faf None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.552177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d71c7-bc7f-41e3-8f4a-ab9c65e20faf None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.552446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d71c7-bc7f-41e3-8f4a-ab9c65e20faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.572595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-8d2d71c7-bc7f-41e3-8f4a-ab9c65e20faf None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:38.573964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8d2d71c7-bc7f-41e3-8f4a-ab9c65e20faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:38.583082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1362/2698] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:38.587279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd677f-5a31-4d8b-9d76-e801ab375b00 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.597756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd677f-5a31-4d8b-9d76-e801ab375b00 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.602322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.602917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.603196 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.603467 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.604107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:38.604414 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.606044 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.618112 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1337/2699] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:26:38.622347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d71c7-bc7f-41e3-8f4a-ab9c65e20faf None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.632073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d71c7-bc7f-41e3-8f4a-ab9c65e20faf None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.636788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ca681-c3ce-4f8a-a250-04ac1f5c7cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.637389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ca681-c3ce-4f8a-a250-04ac1f5c7cfe admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.637702 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ca681-c3ce-4f8a-a250-04ac1f5c7cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.637979 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ca681-c3ce-4f8a-a250-04ac1f5c7cfe admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.638615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ca681-c3ce-4f8a-a250-04ac1f5c7cfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.641100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ca681-c3ce-4f8a-a250-04ac1f5c7cfe admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.667459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1363/2700] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:38 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:38.671368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.681168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8330dd-e1ee-47c7-8f01-1c883f99806b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.736238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87089f-a9e2-4172-a756-50cb0c6541d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.736900 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87089f-a9e2-4172-a756-50cb0c6541d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.737283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87089f-a9e2-4172-a756-50cb0c6541d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.737563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa87089f-a9e2-4172-a756-50cb0c6541d6 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.738413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87089f-a9e2-4172-a756-50cb0c6541d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.740329 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa87089f-a9e2-4172-a756-50cb0c6541d6 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.762493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1338/2701] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:38 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:38.767378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ca681-c3ce-4f8a-a250-04ac1f5c7cfe admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.777672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ca681-c3ce-4f8a-a250-04ac1f5c7cfe admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.782411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29500ba8-5815-470a-ab5e-d6a2be3a8920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.782948 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29500ba8-5815-470a-ab5e-d6a2be3a8920 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.783230 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29500ba8-5815-470a-ab5e-d6a2be3a8920 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.783505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29500ba8-5815-470a-ab5e-d6a2be3a8920 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.784147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29500ba8-5815-470a-ab5e-d6a2be3a8920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.785777 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29500ba8-5815-470a-ab5e-d6a2be3a8920 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.795610 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1364/2702] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:38.799367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87089f-a9e2-4172-a756-50cb0c6541d6 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.809431 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa87089f-a9e2-4172-a756-50cb0c6541d6 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.814627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2fe4f-45ea-48dc-a967-3b7d5bc2a792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.815223 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2fe4f-45ea-48dc-a967-3b7d5bc2a792 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.815502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2fe4f-45ea-48dc-a967-3b7d5bc2a792 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.815774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2fe4f-45ea-48dc-a967-3b7d5bc2a792 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.816410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2fe4f-45ea-48dc-a967-3b7d5bc2a792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.818035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2fe4f-45ea-48dc-a967-3b7d5bc2a792 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.829193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1339/2703] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:38.833329 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29500ba8-5815-470a-ab5e-d6a2be3a8920 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.843279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29500ba8-5815-470a-ab5e-d6a2be3a8920 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.848020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.848694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.848977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.849260 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] PATH_INFO: `/v3/projects/9639394acb914c00b034ba3983bee565/users/228e2723bd124f48a6aa3f19ccb349ef/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.863743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9639394acb914c00b034ba3983bee565, user_id=228e2723bd124f48a6aa3f19ccb349ef, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.866441 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.879568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1365/2704] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:38 2022] PUT /identity/v3/projects/9639394acb914c00b034ba3983bee565/users/228e2723bd124f48a6aa3f19ccb349ef/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:38.890364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0874-be08-42a9-9751-2da225455557 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.890674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0874-be08-42a9-9751-2da225455557 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.890971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0874-be08-42a9-9751-2da225455557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.917724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-185e0874-be08-42a9-9751-2da225455557 None None] MFA Rules not processed for user `228e2723bd124f48a6aa3f19ccb349ef`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:38.918926 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-185e0874-be08-42a9-9751-2da225455557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:38.981404 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1340/2705] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:38.994459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.996205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:39.025146 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:39.083652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589b35d-f72d-49f5-bbd5-bf0b93158da0 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:39.087356 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e61e5-c3ab-40b5-aaf9-53f00d078ce3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d78c1e1b6e46e3ba6271e6743d3c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:39.087956 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e61e5-c3ab-40b5-aaf9-53f00d078ce3 service glance] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:39.088236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e61e5-c3ab-40b5-aaf9-53f00d078ce3 service glance] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:39.088510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e61e5-c3ab-40b5-aaf9-53f00d078ce3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:39.096633 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e61e5-c3ab-40b5-aaf9-53f00d078ce3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:39.098103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290e61e5-c3ab-40b5-aaf9-53f00d078ce3 service glance] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:39.106824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1366/2706] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:47.192145 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e0874-be08-42a9-9751-2da225455557 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.193588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-185e0874-be08-42a9-9751-2da225455557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.246376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:47.262564 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e0874-be08-42a9-9751-2da225455557 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.267615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.268254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.268536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.268809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] PATH_INFO: `/v3/users/baeaefeabbc64616843bfb7a50cd2db7` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.270011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baeaefeabbc64616843bfb7a50cd2db7)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.272027 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.322699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baeaefeabbc64616843bfb7a50cd2db7'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.330353 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baeaefeabbc64616843bfb7a50cd2db7'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.331073 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baeaefeabbc64616843bfb7a50cd2db7'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.343496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baeaefeabbc64616843bfb7a50cd2db7'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.351246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baeaefeabbc64616843bfb7a50cd2db7'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.357378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1341/2707] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:47 2022] DELETE /identity/v3/users/baeaefeabbc64616843bfb7a50cd2db7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:47.362195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166969-f52b-4890-816c-7591f4d2d075 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.362705 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166969-f52b-4890-816c-7591f4d2d075 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.362820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20166969-f52b-4890-816c-7591f4d2d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.381517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-20166969-f52b-4890-816c-7591f4d2d075 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:47.382853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-20166969-f52b-4890-816c-7591f4d2d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.445310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1367/2708] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:47.455668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.457067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.522407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556181f-241b-4e4c-87d8-3472597a54a6 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.525931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834800-400e-412d-877f-bb638c4ae6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.526531 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834800-400e-412d-877f-bb638c4ae6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.526808 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834800-400e-412d-877f-bb638c4ae6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.527083 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834800-400e-412d-877f-bb638c4ae6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.536462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96834800-400e-412d-877f-bb638c4ae6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.538554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96834800-400e-412d-877f-bb638c4ae6c2 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.550370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1342/2709] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:47.823701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166969-f52b-4890-816c-7591f4d2d075 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.825123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-20166969-f52b-4890-816c-7591f4d2d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.846169 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20166969-f52b-4890-816c-7591f4d2d075 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.851178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.851812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.852087 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.852348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] PATH_INFO: `/v3/projects/a889dd814b5f4862ad337877cab6db1b` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.853467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a889dd814b5f4862ad337877cab6db1b)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.855792 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.903309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a889dd814b5f4862ad337877cab6db1b', 'request_id': 'req-e39c8410-5c93-4374-98fd-228b4fec7aa4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.908723 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a889dd814b5f4862ad337877cab6db1b', 'request_id': 'req-e39c8410-5c93-4374-98fd-228b4fec7aa4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.915762 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a889dd814b5f4862ad337877cab6db1b', 'request_id': 'req-e39c8410-5c93-4374-98fd-228b4fec7aa4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.929589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] The token cache is being invalidate because project a889dd814b5f4862ad337877cab6db1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:47.930353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.930524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.931996 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1368/2710] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:47 2022] DELETE /identity/v3/projects/a889dd814b5f4862ad337877cab6db1b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:47.935853 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834800-400e-412d-877f-bb638c4ae6c2 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.946082 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834800-400e-412d-877f-bb638c4ae6c2 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.016291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.017156 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.017572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.018026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] PATH_INFO: `/v3/users/c29010b2219941b9922ef36e1338be40` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.019586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29010b2219941b9922ef36e1338be40)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.022148 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.036110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:48.074170 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29010b2219941b9922ef36e1338be40'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.081536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29010b2219941b9922ef36e1338be40'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.082192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29010b2219941b9922ef36e1338be40'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.095797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29010b2219941b9922ef36e1338be40'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.102715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-1dd5dd08-a0ef-4be7-93d2-d526d5edcd07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29010b2219941b9922ef36e1338be40'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.109106 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1343/2711] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:47 2022] DELETE /identity/v3/users/c29010b2219941b9922ef36e1338be40 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:48.390559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.394931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:48.409178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c8410-5c93-4374-98fd-228b4fec7aa4 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.432438 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:48.471185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.471817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.472094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.472363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] PATH_INFO: `/v3/projects/2c63615298c1491bbc5ffd17ade8a489` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.473563 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c63615298c1491bbc5ffd17ade8a489)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.475511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.525911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c63615298c1491bbc5ffd17ade8a489', 'request_id': 'req-ccced3dc-ae62-4c8c-9724-85427655e205'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.535005 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c63615298c1491bbc5ffd17ade8a489', 'request_id': 'req-ccced3dc-ae62-4c8c-9724-85427655e205'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.541097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c63615298c1491bbc5ffd17ade8a489', 'request_id': 'req-ccced3dc-ae62-4c8c-9724-85427655e205'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.555540 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] The token cache is being invalidate because project 2c63615298c1491bbc5ffd17ade8a489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:48.555832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.556457 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ccced3dc-ae62-4c8c-9724-85427655e205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:48.557988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1369/2712] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:48 2022] DELETE /identity/v3/projects/2c63615298c1491bbc5ffd17ade8a489 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:48.562850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac299ce5-e378-4943-9b0a-f547a5c47e38 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.563371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac299ce5-e378-4943-9b0a-f547a5c47e38 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.563499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac299ce5-e378-4943-9b0a-f547a5c47e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.583821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ac299ce5-e378-4943-9b0a-f547a5c47e38 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:48.585042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ac299ce5-e378-4943-9b0a-f547a5c47e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:48.644830 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1344/2713] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:48.652939 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d790b-ae91-4fd6-bcda-21b3b4fb0d78 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.653226 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d790b-ae91-4fd6-bcda-21b3b4fb0d78 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.653505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d790b-ae91-4fd6-bcda-21b3b4fb0d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.674673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-432d790b-ae91-4fd6-bcda-21b3b4fb0d78 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:48.675905 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-432d790b-ae91-4fd6-bcda-21b3b4fb0d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:48.683878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1370/2714] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:48.688373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac299ce5-e378-4943-9b0a-f547a5c47e38 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.698475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac299ce5-e378-4943-9b0a-f547a5c47e38 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.703102 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.703751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.704033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.704305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.704998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:48.705379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.707138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.719546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1345/2715] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:48.724026 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d790b-ae91-4fd6-bcda-21b3b4fb0d78 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.734112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d790b-ae91-4fd6-bcda-21b3b4fb0d78 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.739468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08798-72c4-42bf-b32c-ee271613b941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.740062 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08798-72c4-42bf-b32c-ee271613b941 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.740339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08798-72c4-42bf-b32c-ee271613b941 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.740606 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08798-72c4-42bf-b32c-ee271613b941 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.741384 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd08798-72c4-42bf-b32c-ee271613b941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.743582 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd08798-72c4-42bf-b32c-ee271613b941 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.771942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1371/2716] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:48 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:48.776084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.786411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0784394e-5561-42da-958c-d13c677d78aa admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.844090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026c035-7d5e-40ef-b1a5-705955024f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.844702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026c035-7d5e-40ef-b1a5-705955024f36 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.844979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026c035-7d5e-40ef-b1a5-705955024f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.845248 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026c035-7d5e-40ef-b1a5-705955024f36 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.845916 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026c035-7d5e-40ef-b1a5-705955024f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.847877 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026c035-7d5e-40ef-b1a5-705955024f36 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.869086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1346/2717] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:48 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:48.873701 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08798-72c4-42bf-b32c-ee271613b941 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.885036 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08798-72c4-42bf-b32c-ee271613b941 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.890231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039ea63-9922-4ffc-93a6-a0a49561c4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.890830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039ea63-9922-4ffc-93a6-a0a49561c4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.891106 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039ea63-9922-4ffc-93a6-a0a49561c4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.891378 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8039ea63-9922-4ffc-93a6-a0a49561c4a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.892082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039ea63-9922-4ffc-93a6-a0a49561c4a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.893757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8039ea63-9922-4ffc-93a6-a0a49561c4a1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.904031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1372/2718] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:48.908502 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026c035-7d5e-40ef-b1a5-705955024f36 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.918898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026c035-7d5e-40ef-b1a5-705955024f36 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.924297 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6691b-764b-4b3b-a8fb-8005f6ea4af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.924945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6691b-764b-4b3b-a8fb-8005f6ea4af8 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.925229 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6691b-764b-4b3b-a8fb-8005f6ea4af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.925500 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6691b-764b-4b3b-a8fb-8005f6ea4af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.926172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6691b-764b-4b3b-a8fb-8005f6ea4af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.927802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6691b-764b-4b3b-a8fb-8005f6ea4af8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.929208 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:48.941049 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1347/2719] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:48.945171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039ea63-9922-4ffc-93a6-a0a49561c4a1 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.956363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8039ea63-9922-4ffc-93a6-a0a49561c4a1 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.961061 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a83cc-cc13-49db-92d1-0997269b8ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.961786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a83cc-cc13-49db-92d1-0997269b8ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.962067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a83cc-cc13-49db-92d1-0997269b8ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.962340 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a83cc-cc13-49db-92d1-0997269b8ac1 admin admin] PATH_INFO: `/v3/projects/0b9e6a4d452040adbfd98d33c4b2c589/users/2d64e80264f044a38decfb4aad54c881/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.977860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a83cc-cc13-49db-92d1-0997269b8ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9e6a4d452040adbfd98d33c4b2c589, user_id=2d64e80264f044a38decfb4aad54c881, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.980544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a83cc-cc13-49db-92d1-0997269b8ac1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.993673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1373/2720] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:48 2022] PUT /identity/v3/projects/0b9e6a4d452040adbfd98d33c4b2c589/users/2d64e80264f044a38decfb4aad54c881/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:49.005350 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6af93-f01c-408d-9737-4a0ae8f446c9 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.005654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6af93-f01c-408d-9737-4a0ae8f446c9 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.006078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6af93-f01c-408d-9737-4a0ae8f446c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.042039 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1cd6af93-f01c-408d-9737-4a0ae8f446c9 None None] MFA Rules not processed for user `2d64e80264f044a38decfb4aad54c881`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:49.043238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1cd6af93-f01c-408d-9737-4a0ae8f446c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:49.108328 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1348/2721] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:49.112907 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a83cc-cc13-49db-92d1-0997269b8ac1 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.164236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:49.177022 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a83cc-cc13-49db-92d1-0997269b8ac1 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.181892 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60ca7c-5c08-42ad-a57e-3f21816121af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.182525 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ca7c-5c08-42ad-a57e-3f21816121af admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.182821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ca7c-5c08-42ad-a57e-3f21816121af admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.183095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ca7c-5c08-42ad-a57e-3f21816121af admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.183733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60ca7c-5c08-42ad-a57e-3f21816121af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.185654 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60ca7c-5c08-42ad-a57e-3f21816121af admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.212551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1374/2722] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:49 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:49.216633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6af93-f01c-408d-9737-4a0ae8f446c9 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.227210 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6af93-f01c-408d-9737-4a0ae8f446c9 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.283437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf406fc3-844e-44ee-9708-22e50ea22a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.284099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf406fc3-844e-44ee-9708-22e50ea22a5d admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.284378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf406fc3-844e-44ee-9708-22e50ea22a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.284652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf406fc3-844e-44ee-9708-22e50ea22a5d admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.285276 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf406fc3-844e-44ee-9708-22e50ea22a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.287243 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf406fc3-844e-44ee-9708-22e50ea22a5d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.313516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1349/2723] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:49 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:49.317327 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60ca7c-5c08-42ad-a57e-3f21816121af admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.328353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60ca7c-5c08-42ad-a57e-3f21816121af admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.332961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc78f-deb2-4624-beb7-b2059e36da5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.333571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc78f-deb2-4624-beb7-b2059e36da5d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.333860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc78f-deb2-4624-beb7-b2059e36da5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.334130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccc78f-deb2-4624-beb7-b2059e36da5d admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.334766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccc78f-deb2-4624-beb7-b2059e36da5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.336343 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccc78f-deb2-4624-beb7-b2059e36da5d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.347496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1375/2724] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:49.351522 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf406fc3-844e-44ee-9708-22e50ea22a5d admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.361707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf406fc3-844e-44ee-9708-22e50ea22a5d admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.366271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1ccec-2fa5-43c3-a48d-485fcfd8a81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.366857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1ccec-2fa5-43c3-a48d-485fcfd8a81a admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.367390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1ccec-2fa5-43c3-a48d-485fcfd8a81a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.367660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1ccec-2fa5-43c3-a48d-485fcfd8a81a admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.368286 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1ccec-2fa5-43c3-a48d-485fcfd8a81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.369908 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1ccec-2fa5-43c3-a48d-485fcfd8a81a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.380240 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1350/2725] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:49.384221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc78f-deb2-4624-beb7-b2059e36da5d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.394725 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccc78f-deb2-4624-beb7-b2059e36da5d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.399447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c015e-63f7-4333-8507-74dbae90547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.400086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c015e-63f7-4333-8507-74dbae90547d admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.400365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c015e-63f7-4333-8507-74dbae90547d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.400634 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c015e-63f7-4333-8507-74dbae90547d admin admin] PATH_INFO: `/v3/projects/f6c95e85a7794cd198cde5393c5edf4e/users/c8424b3df3bd44c38d68d15fe4af263b/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.414945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c015e-63f7-4333-8507-74dbae90547d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c95e85a7794cd198cde5393c5edf4e, user_id=c8424b3df3bd44c38d68d15fe4af263b, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.417557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c015e-63f7-4333-8507-74dbae90547d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.431531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1376/2726] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:49 2022] PUT /identity/v3/projects/f6c95e85a7794cd198cde5393c5edf4e/users/c8424b3df3bd44c38d68d15fe4af263b/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:49.442337 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0568d24-8494-4efb-a8f8-e4bfa39b393d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.442612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0568d24-8494-4efb-a8f8-e4bfa39b393d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.442893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0568d24-8494-4efb-a8f8-e4bfa39b393d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.471305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-a0568d24-8494-4efb-a8f8-e4bfa39b393d None None] MFA Rules not processed for user `c8424b3df3bd44c38d68d15fe4af263b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:49.472516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a0568d24-8494-4efb-a8f8-e4bfa39b393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:49.528459 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:49.539601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1351/2727] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:49.544178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c015e-63f7-4333-8507-74dbae90547d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.606770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c015e-63f7-4333-8507-74dbae90547d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.612117 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bf3fc-9fe5-4898-af02-74a77e7e0b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.612766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf3fc-9fe5-4898-af02-74a77e7e0b3b admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.613045 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf3fc-9fe5-4898-af02-74a77e7e0b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.613310 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bf3fc-9fe5-4898-af02-74a77e7e0b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.613970 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bf3fc-9fe5-4898-af02-74a77e7e0b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.615851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bf3fc-9fe5-4898-af02-74a77e7e0b3b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.648122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1377/2728] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:49 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:49.652233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0568d24-8494-4efb-a8f8-e4bfa39b393d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.662702 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0568d24-8494-4efb-a8f8-e4bfa39b393d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.717270 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc269e-f5b7-475b-bb01-d12399230fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.717848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc269e-f5b7-475b-bb01-d12399230fec admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.718127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc269e-f5b7-475b-bb01-d12399230fec admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.718398 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc269e-f5b7-475b-bb01-d12399230fec admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.719014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc269e-f5b7-475b-bb01-d12399230fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.720921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc269e-f5b7-475b-bb01-d12399230fec admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.742217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1352/2729] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:49 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:49.746617 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bf3fc-9fe5-4898-af02-74a77e7e0b3b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.757527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bf3fc-9fe5-4898-af02-74a77e7e0b3b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.762305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc816-efd0-4618-a894-8f879e3cec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.762902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc816-efd0-4618-a894-8f879e3cec9f admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.763178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc816-efd0-4618-a894-8f879e3cec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.763447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bc816-efd0-4618-a894-8f879e3cec9f admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.764108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bc816-efd0-4618-a894-8f879e3cec9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.765738 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bc816-efd0-4618-a894-8f879e3cec9f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.775915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1378/2730] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:49.780084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc269e-f5b7-475b-bb01-d12399230fec admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.791381 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc269e-f5b7-475b-bb01-d12399230fec admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.796719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5435c6-6b54-4c2f-8fb9-1116c84ab201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.797377 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5435c6-6b54-4c2f-8fb9-1116c84ab201 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.797734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5435c6-6b54-4c2f-8fb9-1116c84ab201 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.798132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5435c6-6b54-4c2f-8fb9-1116c84ab201 admin admin] PATH_INFO: `/v3/projects/4a4e3269bfda4f7cacdc012e56e62144/users/129a94595b954aa1a221d67f1c760d84/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.816838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5435c6-6b54-4c2f-8fb9-1116c84ab201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4e3269bfda4f7cacdc012e56e62144, user_id=129a94595b954aa1a221d67f1c760d84, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.819810 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5435c6-6b54-4c2f-8fb9-1116c84ab201 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.832415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1353/2731] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:49 2022] PUT /identity/v3/projects/4a4e3269bfda4f7cacdc012e56e62144/users/129a94595b954aa1a221d67f1c760d84/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:49.836282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc816-efd0-4618-a894-8f879e3cec9f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.846564 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bc816-efd0-4618-a894-8f879e3cec9f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.901748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914ff2f-2582-43f8-8e3d-7785e7b1d10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.902353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914ff2f-2582-43f8-8e3d-7785e7b1d10e admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.902630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914ff2f-2582-43f8-8e3d-7785e7b1d10e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.902898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914ff2f-2582-43f8-8e3d-7785e7b1d10e admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.903534 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914ff2f-2582-43f8-8e3d-7785e7b1d10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.905664 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914ff2f-2582-43f8-8e3d-7785e7b1d10e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.915023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1379/2732] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:49.919192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5435c6-6b54-4c2f-8fb9-1116c84ab201 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.929967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5435c6-6b54-4c2f-8fb9-1116c84ab201 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.934763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.935481 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.935853 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.936131 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] PATH_INFO: `/v3/projects/4a4e3269bfda4f7cacdc012e56e62144/users/129a94595b954aa1a221d67f1c760d84/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.938124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4e3269bfda4f7cacdc012e56e62144, user_id=129a94595b954aa1a221d67f1c760d84, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.940738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.951110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1354/2733] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:49 2022] PUT /identity/v3/projects/4a4e3269bfda4f7cacdc012e56e62144/users/129a94595b954aa1a221d67f1c760d84/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:49.961292 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185afe8-982e-4716-9f55-b5d3bef4deea None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.961571 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185afe8-982e-4716-9f55-b5d3bef4deea None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.961878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185afe8-982e-4716-9f55-b5d3bef4deea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.982312 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:49.991004 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-1185afe8-982e-4716-9f55-b5d3bef4deea None None] MFA Rules not processed for user `129a94595b954aa1a221d67f1c760d84`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:49.992669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1185afe8-982e-4716-9f55-b5d3bef4deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:50.059933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1380/2734] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:49 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:50.070894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:50.072671 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:50.135612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b0b3b-c298-404e-bd98-9041ebe0a1fb admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:50.139250 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430efa53-8e14-4896-bd14-8e169af88a10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:50.139960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430efa53-8e14-4896-bd14-8e169af88a10 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:50.140236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430efa53-8e14-4896-bd14-8e169af88a10 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:50.140508 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430efa53-8e14-4896-bd14-8e169af88a10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:50.149512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430efa53-8e14-4896-bd14-8e169af88a10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:50.150971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430efa53-8e14-4896-bd14-8e169af88a10 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:50.160688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1355/2735] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:50.206473 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185afe8-982e-4716-9f55-b5d3bef4deea None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:50.207849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-1185afe8-982e-4716-9f55-b5d3bef4deea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:50.220060 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185afe8-982e-4716-9f55-b5d3bef4deea None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:50.223626 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50724df-b318-4537-ae58-41293f6e9a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:50.224213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50724df-b318-4537-ae58-41293f6e9a3b service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:50.224487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50724df-b318-4537-ae58-41293f6e9a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:50.224755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50724df-b318-4537-ae58-41293f6e9a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:50.281417 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50724df-b318-4537-ae58-41293f6e9a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:50.282835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50724df-b318-4537-ae58-41293f6e9a3b service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:50.291651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1381/2736] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:53.164677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430efa53-8e14-4896-bd14-8e169af88a10 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:53.210479 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:53.228491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430efa53-8e14-4896-bd14-8e169af88a10 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:53.232967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:53.234252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:53.234631 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:53.235025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] PATH_INFO: `/v3/users/2d64e80264f044a38decfb4aad54c881` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:53.236318 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d64e80264f044a38decfb4aad54c881)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:53.238293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:53.286956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d64e80264f044a38decfb4aad54c881'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.294077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d64e80264f044a38decfb4aad54c881'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.294727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d64e80264f044a38decfb4aad54c881'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.306722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d64e80264f044a38decfb4aad54c881'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.313762 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d64e80264f044a38decfb4aad54c881'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.319554 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1356/2737] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:53 2022] DELETE /identity/v3/users/2d64e80264f044a38decfb4aad54c881 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:53.324228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e27b6-0962-49ab-a0fc-20f2e25f30ce None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:53.324713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e27b6-0962-49ab-a0fc-20f2e25f30ce None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:53.324836 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e27b6-0962-49ab-a0fc-20f2e25f30ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:53.344802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-3a0e27b6-0962-49ab-a0fc-20f2e25f30ce None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:53.346031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3a0e27b6-0962-49ab-a0fc-20f2e25f30ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:53.402927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1382/2738] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:53.413128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:53.414633 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:53.470330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27135799-546d-48f2-afb3-dd87bd2df1f7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:53.473934 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86243f46-6d87-4674-b02b-b8eeca2924f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:53.474526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86243f46-6d87-4674-b02b-b8eeca2924f8 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:53.474806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86243f46-6d87-4674-b02b-b8eeca2924f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:53.475075 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86243f46-6d87-4674-b02b-b8eeca2924f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:53.483904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86243f46-6d87-4674-b02b-b8eeca2924f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:53.485322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86243f46-6d87-4674-b02b-b8eeca2924f8 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:53.494751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1357/2739] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:53.760878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e27b6-0962-49ab-a0fc-20f2e25f30ce None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:53.762320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3a0e27b6-0962-49ab-a0fc-20f2e25f30ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:53.775301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e27b6-0962-49ab-a0fc-20f2e25f30ce None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:53.780272 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:53.780995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:53.781369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:53.781669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] PATH_INFO: `/v3/projects/0b9e6a4d452040adbfd98d33c4b2c589` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:53.782890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9e6a4d452040adbfd98d33c4b2c589)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:53.785255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:53.819585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:53.833526 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9e6a4d452040adbfd98d33c4b2c589', 'request_id': 'req-5ea97423-898e-49a4-be56-d26e311551dd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.839733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9e6a4d452040adbfd98d33c4b2c589', 'request_id': 'req-5ea97423-898e-49a4-be56-d26e311551dd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.846439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9e6a4d452040adbfd98d33c4b2c589', 'request_id': 'req-5ea97423-898e-49a4-be56-d26e311551dd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.860532 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] The token cache is being invalidate because project 0b9e6a4d452040adbfd98d33c4b2c589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:53.860822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.861462 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.862990 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1383/2740] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:53 2022] DELETE /identity/v3/projects/0b9e6a4d452040adbfd98d33c4b2c589 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:53.867264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86243f46-6d87-4674-b02b-b8eeca2924f8 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:53.877726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86243f46-6d87-4674-b02b-b8eeca2924f8 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:53.932690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:53.933315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:53.933598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:53.933895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] PATH_INFO: `/v3/users/c8424b3df3bd44c38d68d15fe4af263b` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:53.935003 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8424b3df3bd44c38d68d15fe4af263b)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:53.937447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:53.949428 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:53.984257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8424b3df3bd44c38d68d15fe4af263b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.991434 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8424b3df3bd44c38d68d15fe4af263b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:53.992463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8424b3df3bd44c38d68d15fe4af263b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.004873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8424b3df3bd44c38d68d15fe4af263b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.011782 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8424b3df3bd44c38d68d15fe4af263b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.017940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1358/2741] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:53 2022] DELETE /identity/v3/users/c8424b3df3bd44c38d68d15fe4af263b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:54.299082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:54.301033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:54.317847 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea97423-898e-49a4-be56-d26e311551dd admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:54.391144 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:54.392524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:54.393012 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:54.393489 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] PATH_INFO: `/v3/projects/f6c95e85a7794cd198cde5393c5edf4e` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:54.394868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c95e85a7794cd198cde5393c5edf4e)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:54.397136 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:54.453141 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c95e85a7794cd198cde5393c5edf4e', 'request_id': 'req-5b22227b-a943-4890-ac30-d02588a5dfb6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.458964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c95e85a7794cd198cde5393c5edf4e', 'request_id': 'req-5b22227b-a943-4890-ac30-d02588a5dfb6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.465805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c95e85a7794cd198cde5393c5edf4e', 'request_id': 'req-5b22227b-a943-4890-ac30-d02588a5dfb6'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.480419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] The token cache is being invalidate because project f6c95e85a7794cd198cde5393c5edf4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:54.480715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.481699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.484177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1384/2742] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:54 2022] DELETE /identity/v3/projects/f6c95e85a7794cd198cde5393c5edf4e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:54.488427 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:54.490333 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:54.504340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b1b84-5ecc-4f18-9dec-56054d685d1f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:54.563460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:54.564133 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:54.564443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:54.564789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] PATH_INFO: `/v3/users/129a94595b954aa1a221d67f1c760d84` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:54.566010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=129a94595b954aa1a221d67f1c760d84)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:54.567961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:54.626060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '129a94595b954aa1a221d67f1c760d84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.634605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '129a94595b954aa1a221d67f1c760d84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.635260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '129a94595b954aa1a221d67f1c760d84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.648946 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '129a94595b954aa1a221d67f1c760d84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.669717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e4ed4b86-c2eb-4e5a-b14f-0773f87aa7c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '129a94595b954aa1a221d67f1c760d84'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:54.676450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1359/2743] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:54 2022] DELETE /identity/v3/users/129a94595b954aa1a221d67f1c760d84 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:54.949681 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:54.951229 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:54.965003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22227b-a943-4890-ac30-d02588a5dfb6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.010393 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:55.025119 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.025817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.026083 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.026361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] PATH_INFO: `/v3/projects/4a4e3269bfda4f7cacdc012e56e62144` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.027485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4e3269bfda4f7cacdc012e56e62144)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.030621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.077622 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4e3269bfda4f7cacdc012e56e62144', 'request_id': 'req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:55.083070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4e3269bfda4f7cacdc012e56e62144', 'request_id': 'req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:55.089015 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4e3269bfda4f7cacdc012e56e62144', 'request_id': 'req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:55.101424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] The token cache is being invalidate because project 4a4e3269bfda4f7cacdc012e56e62144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:55.101749 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:55.102371 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-a1cb51ba-bb8e-4ac2-aab6-29658f78371e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:55.103828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1385/2744] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:54 2022] DELETE /identity/v3/projects/4a4e3269bfda4f7cacdc012e56e62144 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:55.109726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05f6b-9c30-45b4-887b-15d6881e0050 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.110010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05f6b-9c30-45b4-887b-15d6881e0050 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.110283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05f6b-9c30-45b4-887b-15d6881e0050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.130933 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b7a05f6b-9c30-45b4-887b-15d6881e0050 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:55.132125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b7a05f6b-9c30-45b4-887b-15d6881e0050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:55.136566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:55.192842 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1360/2745] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:55.256951 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26943-8670-46cc-a149-95f85e7cbbab None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.257252 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26943-8670-46cc-a149-95f85e7cbbab None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.257579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26943-8670-46cc-a149-95f85e7cbbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.277677 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-75c26943-8670-46cc-a149-95f85e7cbbab None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:55.278873 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-75c26943-8670-46cc-a149-95f85e7cbbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:55.286628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1386/2746] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:55.291031 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05f6b-9c30-45b4-887b-15d6881e0050 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.300807 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05f6b-9c30-45b4-887b-15d6881e0050 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.305424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.306058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.306340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.306638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.307278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:55.307584 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.309176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.320989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1361/2747] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:55.325334 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c26943-8670-46cc-a149-95f85e7cbbab None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.334980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c26943-8670-46cc-a149-95f85e7cbbab None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.339760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6a1c-f602-4b03-8b02-cba7ee2bf79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.340351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6a1c-f602-4b03-8b02-cba7ee2bf79f admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.340627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6a1c-f602-4b03-8b02-cba7ee2bf79f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.340941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6a1c-f602-4b03-8b02-cba7ee2bf79f admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.341575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d6a1c-f602-4b03-8b02-cba7ee2bf79f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.343537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d6a1c-f602-4b03-8b02-cba7ee2bf79f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.370215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1387/2748] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:55 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:55.374734 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.384503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626444a4-ce7a-47c8-97d2-1fbf3e7c77f6 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.439640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be040a6e-dd12-4b3c-b1b8-c38c07b19d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.440304 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be040a6e-dd12-4b3c-b1b8-c38c07b19d21 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.440587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be040a6e-dd12-4b3c-b1b8-c38c07b19d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.440862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be040a6e-dd12-4b3c-b1b8-c38c07b19d21 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.441497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be040a6e-dd12-4b3c-b1b8-c38c07b19d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.443463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be040a6e-dd12-4b3c-b1b8-c38c07b19d21 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.464166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1362/2749] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:55 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:55.468511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6a1c-f602-4b03-8b02-cba7ee2bf79f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.478512 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6a1c-f602-4b03-8b02-cba7ee2bf79f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.483625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6333197-b243-4793-b3e2-7b36504fd213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.484213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6333197-b243-4793-b3e2-7b36504fd213 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.484485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6333197-b243-4793-b3e2-7b36504fd213 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.484753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6333197-b243-4793-b3e2-7b36504fd213 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.485407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6333197-b243-4793-b3e2-7b36504fd213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.487027 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6333197-b243-4793-b3e2-7b36504fd213 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.497125 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1388/2750] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:55.501558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be040a6e-dd12-4b3c-b1b8-c38c07b19d21 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.511226 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be040a6e-dd12-4b3c-b1b8-c38c07b19d21 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.515805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c8857-3f37-4cd8-8ce2-9e3ea7b05924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.516386 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c8857-3f37-4cd8-8ce2-9e3ea7b05924 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.516662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c8857-3f37-4cd8-8ce2-9e3ea7b05924 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.516935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c8857-3f37-4cd8-8ce2-9e3ea7b05924 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.517559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c8857-3f37-4cd8-8ce2-9e3ea7b05924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.519172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c8857-3f37-4cd8-8ce2-9e3ea7b05924 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.529424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1363/2751] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:55.533264 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6333197-b243-4793-b3e2-7b36504fd213 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.543432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6333197-b243-4793-b3e2-7b36504fd213 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.548565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba51c6-e801-4767-9a02-d321b24d7a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.549204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba51c6-e801-4767-9a02-d321b24d7a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.549479 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba51c6-e801-4767-9a02-d321b24d7a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.549781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba51c6-e801-4767-9a02-d321b24d7a98 admin admin] PATH_INFO: `/v3/projects/adbc7253e5ef45b1912cfc9b5b0be995/users/3ac145e04afb4d8fa03579d0d2301e79/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.565063 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba51c6-e801-4767-9a02-d321b24d7a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adbc7253e5ef45b1912cfc9b5b0be995, user_id=3ac145e04afb4d8fa03579d0d2301e79, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.567788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba51c6-e801-4767-9a02-d321b24d7a98 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.580250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1389/2752] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:55 2022] PUT /identity/v3/projects/adbc7253e5ef45b1912cfc9b5b0be995/users/3ac145e04afb4d8fa03579d0d2301e79/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:55.590825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795697f-a309-4bde-806a-fc18aa341c15 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.591115 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795697f-a309-4bde-806a-fc18aa341c15 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.591394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795697f-a309-4bde-806a-fc18aa341c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.620200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b795697f-a309-4bde-806a-fc18aa341c15 None None] MFA Rules not processed for user `3ac145e04afb4d8fa03579d0d2301e79`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:55.621430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b795697f-a309-4bde-806a-fc18aa341c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:55.685982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1364/2753] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:55.691033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba51c6-e801-4767-9a02-d321b24d7a98 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.755033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba51c6-e801-4767-9a02-d321b24d7a98 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.759652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990691-1318-4908-a382-04b6f18c90e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.760248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990691-1318-4908-a382-04b6f18c90e6 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.760523 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990691-1318-4908-a382-04b6f18c90e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.760795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8990691-1318-4908-a382-04b6f18c90e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.761414 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990691-1318-4908-a382-04b6f18c90e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.763348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8990691-1318-4908-a382-04b6f18c90e6 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.783283 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:55.790075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1390/2754] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:55 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:55.794136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795697f-a309-4bde-806a-fc18aa341c15 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.801231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:55.805499 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795697f-a309-4bde-806a-fc18aa341c15 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.860393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ced57-6c11-410a-97f5-28508e41037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.861000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ced57-6c11-410a-97f5-28508e41037c admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.861284 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ced57-6c11-410a-97f5-28508e41037c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.861559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ced57-6c11-410a-97f5-28508e41037c admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.862225 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ced57-6c11-410a-97f5-28508e41037c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.864147 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ced57-6c11-410a-97f5-28508e41037c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.885453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1365/2755] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:55 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:55.889565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990691-1318-4908-a382-04b6f18c90e6 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.899798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8990691-1318-4908-a382-04b6f18c90e6 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.904370 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fd6f7-89f1-4ff1-a1a8-d2cb863d9a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.905498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fd6f7-89f1-4ff1-a1a8-d2cb863d9a32 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.905805 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fd6f7-89f1-4ff1-a1a8-d2cb863d9a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.906078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fd6f7-89f1-4ff1-a1a8-d2cb863d9a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.906706 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fd6f7-89f1-4ff1-a1a8-d2cb863d9a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.908279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fd6f7-89f1-4ff1-a1a8-d2cb863d9a32 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.918561 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1391/2756] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:55.923193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ced57-6c11-410a-97f5-28508e41037c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.932890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ced57-6c11-410a-97f5-28508e41037c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:55.937439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69974cb8-9b82-4f46-9b65-609b8d46dbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:55.938125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69974cb8-9b82-4f46-9b65-609b8d46dbfe admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:55.938394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69974cb8-9b82-4f46-9b65-609b8d46dbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:55.938664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69974cb8-9b82-4f46-9b65-609b8d46dbfe admin admin] PATH_INFO: `/v3/projects/3f69a4665afd4d3baaa927ec06a9f588/users/7c1b23844318474ab389ac5ded5067df/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:55.952870 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69974cb8-9b82-4f46-9b65-609b8d46dbfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f69a4665afd4d3baaa927ec06a9f588, user_id=7c1b23844318474ab389ac5ded5067df, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:55.955569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69974cb8-9b82-4f46-9b65-609b8d46dbfe admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:55.969294 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1366/2757] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:55 2022] PUT /identity/v3/projects/3f69a4665afd4d3baaa927ec06a9f588/users/7c1b23844318474ab389ac5ded5067df/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:55.973218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fd6f7-89f1-4ff1-a1a8-d2cb863d9a32 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:55.983613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fd6f7-89f1-4ff1-a1a8-d2cb863d9a32 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:56.040306 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001b676-1940-4198-af35-b62ba746281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:56.040913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b676-1940-4198-af35-b62ba746281c admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:56.041194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b676-1940-4198-af35-b62ba746281c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:56.041468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001b676-1940-4198-af35-b62ba746281c admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:56.042138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001b676-1940-4198-af35-b62ba746281c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:56.043735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001b676-1940-4198-af35-b62ba746281c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:56.053286 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1392/2758] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:56.057601 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69974cb8-9b82-4f46-9b65-609b8d46dbfe admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:56.068214 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69974cb8-9b82-4f46-9b65-609b8d46dbfe admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:56.072858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:56.073496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:56.073821 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:56.074094 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] PATH_INFO: `/v3/projects/3f69a4665afd4d3baaa927ec06a9f588/users/7c1b23844318474ab389ac5ded5067df/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:56.075979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f69a4665afd4d3baaa927ec06a9f588, user_id=7c1b23844318474ab389ac5ded5067df, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:56.079292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:56.089218 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1367/2759] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:56 2022] PUT /identity/v3/projects/3f69a4665afd4d3baaa927ec06a9f588/users/7c1b23844318474ab389ac5ded5067df/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:56.098983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86bf54-e834-44f7-92be-9e7c10855781 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:56.099273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86bf54-e834-44f7-92be-9e7c10855781 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:56.099543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86bf54-e834-44f7-92be-9e7c10855781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:56.127929 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-9a86bf54-e834-44f7-92be-9e7c10855781 None None] MFA Rules not processed for user `7c1b23844318474ab389ac5ded5067df`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:56.129130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9a86bf54-e834-44f7-92be-9e7c10855781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:56.195743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1393/2760] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:56.207433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:56.209153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:56.265548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a50a6-a12b-4371-927d-421504b69054 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:56.269178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714941b4-523c-4dc7-b91c-a0695d6ce881 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:56.269791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714941b4-523c-4dc7-b91c-a0695d6ce881 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:56.270071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714941b4-523c-4dc7-b91c-a0695d6ce881 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:56.270342 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714941b4-523c-4dc7-b91c-a0695d6ce881 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:56.279124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714941b4-523c-4dc7-b91c-a0695d6ce881 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:56.280567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714941b4-523c-4dc7-b91c-a0695d6ce881 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:56.289791 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1368/2761] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:56.652913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86bf54-e834-44f7-92be-9e7c10855781 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:56.713487 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86bf54-e834-44f7-92be-9e7c10855781 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:56.718594 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:56.719218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:56.719497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:56.719770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] PATH_INFO: `/v3/users/3ac145e04afb4d8fa03579d0d2301e79` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:56.720888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac145e04afb4d8fa03579d0d2301e79)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:56.722844 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:56.736732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:56.779210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac145e04afb4d8fa03579d0d2301e79'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:56.787160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac145e04afb4d8fa03579d0d2301e79'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:56.787854 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac145e04afb4d8fa03579d0d2301e79'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:56.800193 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac145e04afb4d8fa03579d0d2301e79'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:56.806904 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac145e04afb4d8fa03579d0d2301e79'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:56.812619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1394/2762] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:56 2022] DELETE /identity/v3/users/3ac145e04afb4d8fa03579d0d2301e79 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:56.817059 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204ca1-1fe3-4bcc-b5c4-beec67c1947c None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:56.817547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204ca1-1fe3-4bcc-b5c4-beec67c1947c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:56.817678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9204ca1-1fe3-4bcc-b5c4-beec67c1947c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:56.836025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f9204ca1-1fe3-4bcc-b5c4-beec67c1947c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:56.837224 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f9204ca1-1fe3-4bcc-b5c4-beec67c1947c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:56.887254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:56.896470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1369/2763] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:56.906666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:56.908058 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:56.963817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7e5e5-d371-4a45-bb45-ba5873021e5e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:56.968006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc54dce-5aa1-4ae6-9ed0-281077c5777f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:56.968601 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc54dce-5aa1-4ae6-9ed0-281077c5777f service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:56.968880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc54dce-5aa1-4ae6-9ed0-281077c5777f service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:56.969154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc54dce-5aa1-4ae6-9ed0-281077c5777f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:56.978152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc54dce-5aa1-4ae6-9ed0-281077c5777f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:56.979586 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc54dce-5aa1-4ae6-9ed0-281077c5777f service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:56.989577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1395/2764] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:57.253164 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9204ca1-1fe3-4bcc-b5c4-beec67c1947c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:57.254637 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f9204ca1-1fe3-4bcc-b5c4-beec67c1947c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:57.272362 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9204ca1-1fe3-4bcc-b5c4-beec67c1947c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:57.277699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:57.278393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:57.278615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:57.278923 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] PATH_INFO: `/v3/projects/adbc7253e5ef45b1912cfc9b5b0be995` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:57.280854 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adbc7253e5ef45b1912cfc9b5b0be995)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:57.282736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:57.329652 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adbc7253e5ef45b1912cfc9b5b0be995', 'request_id': 'req-e624ac9f-2cd5-440f-a087-20d83240aba1'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.335445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adbc7253e5ef45b1912cfc9b5b0be995', 'request_id': 'req-e624ac9f-2cd5-440f-a087-20d83240aba1'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.341815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adbc7253e5ef45b1912cfc9b5b0be995', 'request_id': 'req-e624ac9f-2cd5-440f-a087-20d83240aba1'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.355568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] The token cache is being invalidate because project adbc7253e5ef45b1912cfc9b5b0be995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:57.355856 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.356472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.357996 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1370/2765] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:57 2022] DELETE /identity/v3/projects/adbc7253e5ef45b1912cfc9b5b0be995 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:57.362178 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc54dce-5aa1-4ae6-9ed0-281077c5777f service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:57.372034 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc54dce-5aa1-4ae6-9ed0-281077c5777f service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:57.428642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:57.429269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:57.429549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:57.429853 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] PATH_INFO: `/v3/users/7c1b23844318474ab389ac5ded5067df` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:57.430971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1b23844318474ab389ac5ded5067df)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:57.432877 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:57.470190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:57.481079 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1b23844318474ab389ac5ded5067df'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.488211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1b23844318474ab389ac5ded5067df'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.488859 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1b23844318474ab389ac5ded5067df'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.501499 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1b23844318474ab389ac5ded5067df'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.508168 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-412154ac-384d-4096-8388-75cdbcb821d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1b23844318474ab389ac5ded5067df'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.513922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1396/2766] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:57 2022] DELETE /identity/v3/users/7c1b23844318474ab389ac5ded5067df => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:57.786282 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:57.787668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:57.803216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624ac9f-2cd5-440f-a087-20d83240aba1 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:57.862007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:57.862668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:57.862913 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:57.863189 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] PATH_INFO: `/v3/projects/3f69a4665afd4d3baaa927ec06a9f588` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:57.864328 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f69a4665afd4d3baaa927ec06a9f588)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:57.866281 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:57.912169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f69a4665afd4d3baaa927ec06a9f588', 'request_id': 'req-ef84b955-e75c-4454-8a11-1399a4a989dc'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.917596 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f69a4665afd4d3baaa927ec06a9f588', 'request_id': 'req-ef84b955-e75c-4454-8a11-1399a4a989dc'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.924159 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f69a4665afd4d3baaa927ec06a9f588', 'request_id': 'req-ef84b955-e75c-4454-8a11-1399a4a989dc'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.936104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] The token cache is being invalidate because project 3f69a4665afd4d3baaa927ec06a9f588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:57.936399 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.937566 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-ef84b955-e75c-4454-8a11-1399a4a989dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:57.939058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1371/2767] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:57 2022] DELETE /identity/v3/projects/3f69a4665afd4d3baaa927ec06a9f588 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:57.945189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558bc7-d116-4dfa-8c13-70069d16b873 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:57.945781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558bc7-d116-4dfa-8c13-70069d16b873 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:57.945903 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92558bc7-d116-4dfa-8c13-70069d16b873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:57.965082 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-92558bc7-d116-4dfa-8c13-70069d16b873 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:57.966295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-92558bc7-d116-4dfa-8c13-70069d16b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.027431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1397/2768] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:58.035249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159fdc7-d494-4b1c-b729-26f545151ab3 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.035476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159fdc7-d494-4b1c-b729-26f545151ab3 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.035753 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159fdc7-d494-4b1c-b729-26f545151ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.055613 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-e159fdc7-d494-4b1c-b729-26f545151ab3 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:58.056805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e159fdc7-d494-4b1c-b729-26f545151ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.065053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1372/2769] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:58.069244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92558bc7-d116-4dfa-8c13-70069d16b873 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.079248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92558bc7-d116-4dfa-8c13-70069d16b873 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.083899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.084500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.084783 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.085055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.085759 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:58.086084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.087729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.100243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1398/2770] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:58.104857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159fdc7-d494-4b1c-b729-26f545151ab3 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.114859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159fdc7-d494-4b1c-b729-26f545151ab3 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.119839 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544202d-ad6f-4b77-baf0-94f3b3124ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.120441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544202d-ad6f-4b77-baf0-94f3b3124ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.120724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544202d-ad6f-4b77-baf0-94f3b3124ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.120997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544202d-ad6f-4b77-baf0-94f3b3124ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.121664 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544202d-ad6f-4b77-baf0-94f3b3124ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.123668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544202d-ad6f-4b77-baf0-94f3b3124ab3 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.128199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:58.152395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1373/2771] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:58.157397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.167247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e49200-185f-4d2c-a1d2-7964334f1056 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.221493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1f7c5-bf08-4bc8-ad0c-b3e8cf789f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.222111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1f7c5-bf08-4bc8-ad0c-b3e8cf789f9c admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.222386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1f7c5-bf08-4bc8-ad0c-b3e8cf789f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.222653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1f7c5-bf08-4bc8-ad0c-b3e8cf789f9c admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.223275 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1f7c5-bf08-4bc8-ad0c-b3e8cf789f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.225160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1f7c5-bf08-4bc8-ad0c-b3e8cf789f9c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.248429 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1399/2772] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:58 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:58.252670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544202d-ad6f-4b77-baf0-94f3b3124ab3 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.262835 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544202d-ad6f-4b77-baf0-94f3b3124ab3 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.267457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b65011-f3e0-4c8f-bec1-fb62d76c57df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.268045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b65011-f3e0-4c8f-bec1-fb62d76c57df admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.268322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b65011-f3e0-4c8f-bec1-fb62d76c57df admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.268588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b65011-f3e0-4c8f-bec1-fb62d76c57df admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.269216 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b65011-f3e0-4c8f-bec1-fb62d76c57df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.270837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b65011-f3e0-4c8f-bec1-fb62d76c57df admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.280895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1374/2773] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:58.284589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1f7c5-bf08-4bc8-ad0c-b3e8cf789f9c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.294392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1f7c5-bf08-4bc8-ad0c-b3e8cf789f9c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.299549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2789a2-3ddb-4be2-8cb8-76d43b997fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.300129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2789a2-3ddb-4be2-8cb8-76d43b997fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.300403 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2789a2-3ddb-4be2-8cb8-76d43b997fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.300673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2789a2-3ddb-4be2-8cb8-76d43b997fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.301296 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2789a2-3ddb-4be2-8cb8-76d43b997fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.302913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2789a2-3ddb-4be2-8cb8-76d43b997fd7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.313203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1400/2774] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:58.317177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b65011-f3e0-4c8f-bec1-fb62d76c57df admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.327107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b65011-f3e0-4c8f-bec1-fb62d76c57df admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.331800 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.332430 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.332704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.332975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] PATH_INFO: `/v3/projects/30f40f7a316a4ce08a20d3ee47ba5b4b/users/d0075013f55544c597727bf4e34def5b/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.350574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f40f7a316a4ce08a20d3ee47ba5b4b, user_id=d0075013f55544c597727bf4e34def5b, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.353289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.370293 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1375/2775] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:58 2022] PUT /identity/v3/projects/30f40f7a316a4ce08a20d3ee47ba5b4b/users/d0075013f55544c597727bf4e34def5b/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:58.380335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128a7db-4ea1-4ef7-b4cb-695765139f19 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.380642 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128a7db-4ea1-4ef7-b4cb-695765139f19 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.380919 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128a7db-4ea1-4ef7-b4cb-695765139f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.408754 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5128a7db-4ea1-4ef7-b4cb-695765139f19 None None] MFA Rules not processed for user `d0075013f55544c597727bf4e34def5b`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:58.410006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5128a7db-4ea1-4ef7-b4cb-695765139f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.414936 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:58.474961 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1401/2776] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:58.486895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.488562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.544360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad954586-ffa7-4571-929f-73bb2894986f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.548169 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129d7b5-ae5d-4e90-af22-5ba467e93b47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.548700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d7b5-ae5d-4e90-af22-5ba467e93b47 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.548978 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d7b5-ae5d-4e90-af22-5ba467e93b47 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.549247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129d7b5-ae5d-4e90-af22-5ba467e93b47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.557255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129d7b5-ae5d-4e90-af22-5ba467e93b47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.558700 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129d7b5-ae5d-4e90-af22-5ba467e93b47 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.567103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1376/2777] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:58.591498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128a7db-4ea1-4ef7-b4cb-695765139f19 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.652078 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128a7db-4ea1-4ef7-b4cb-695765139f19 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.657308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.657960 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.658239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.658511 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] PATH_INFO: `/v3/users/d0075013f55544c597727bf4e34def5b` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.659640 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0075013f55544c597727bf4e34def5b)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.661551 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.707294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0075013f55544c597727bf4e34def5b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.714585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0075013f55544c597727bf4e34def5b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.715241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0075013f55544c597727bf4e34def5b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.727980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0075013f55544c597727bf4e34def5b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.734959 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0075013f55544c597727bf4e34def5b'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.741964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1402/2778] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:58 2022] DELETE /identity/v3/users/d0075013f55544c597727bf4e34def5b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:58.746464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e3ee5-d3a2-4bb6-902a-edb84469e52c None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.746751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e3ee5-d3a2-4bb6-902a-edb84469e52c None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.747150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1e3ee5-d3a2-4bb6-902a-edb84469e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.766176 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-7d1e3ee5-d3a2-4bb6-902a-edb84469e52c None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:58.767369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7d1e3ee5-d3a2-4bb6-902a-edb84469e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.823432 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1377/2779] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:58.833313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.834785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.868255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:58.898320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb257806-3f95-4dfb-bc4a-4d912d06bc1b admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.901943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f1396-86cb-4245-968a-a47c4d93d4d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.902545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f1396-86cb-4245-968a-a47c4d93d4d5 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.902823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f1396-86cb-4245-968a-a47c4d93d4d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.903093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f1396-86cb-4245-968a-a47c4d93d4d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.911931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f1396-86cb-4245-968a-a47c4d93d4d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.913347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f1396-86cb-4245-968a-a47c4d93d4d5 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.922686 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1403/2780] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:59.438771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e3ee5-d3a2-4bb6-902a-edb84469e52c None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.440357 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7d1e3ee5-d3a2-4bb6-902a-edb84469e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:59.453993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1e3ee5-d3a2-4bb6-902a-edb84469e52c None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.458717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.459352 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.459639 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.459925 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] PATH_INFO: `/v3/projects/30f40f7a316a4ce08a20d3ee47ba5b4b` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.461071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f40f7a316a4ce08a20d3ee47ba5b4b)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.463035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.508518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:59.509921 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f40f7a316a4ce08a20d3ee47ba5b4b', 'request_id': 'req-04ecc00a-59e7-4c50-8129-27ba648a85d5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.517204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f40f7a316a4ce08a20d3ee47ba5b4b', 'request_id': 'req-04ecc00a-59e7-4c50-8129-27ba648a85d5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.523472 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f40f7a316a4ce08a20d3ee47ba5b4b', 'request_id': 'req-04ecc00a-59e7-4c50-8129-27ba648a85d5'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.535935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] The token cache is being invalidate because project 30f40f7a316a4ce08a20d3ee47ba5b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:59.536203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.536827 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-04ecc00a-59e7-4c50-8129-27ba648a85d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.538335 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1378/2781] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:59 2022] DELETE /identity/v3/projects/30f40f7a316a4ce08a20d3ee47ba5b4b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:59.543444 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898daf13-308c-48e7-b6bb-0c6fd9dbac3d None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.543917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898daf13-308c-48e7-b6bb-0c6fd9dbac3d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.544021 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898daf13-308c-48e7-b6bb-0c6fd9dbac3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.563038 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-898daf13-308c-48e7-b6bb-0c6fd9dbac3d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:59.564220 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-898daf13-308c-48e7-b6bb-0c6fd9dbac3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:59.622361 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1404/2782] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:59.631461 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6dc1-7bca-43aa-93a2-126700b23f24 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.631754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6dc1-7bca-43aa-93a2-126700b23f24 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.632034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc6dc1-7bca-43aa-93a2-126700b23f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.651918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-23bc6dc1-7bca-43aa-93a2-126700b23f24 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:59.653048 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-23bc6dc1-7bca-43aa-93a2-126700b23f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:59.660892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1379/2783] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:59.664898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898daf13-308c-48e7-b6bb-0c6fd9dbac3d None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.674631 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898daf13-308c-48e7-b6bb-0c6fd9dbac3d None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.679101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.679700 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.679971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.680238 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.681044 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:59.681406 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.683212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.695333 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1405/2784] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:59.699774 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc6dc1-7bca-43aa-93a2-126700b23f24 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.709521 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc6dc1-7bca-43aa-93a2-126700b23f24 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.714287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d2051-6071-4dc1-93e0-4b4d9b91249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.714886 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d2051-6071-4dc1-93e0-4b4d9b91249f admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.715168 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d2051-6071-4dc1-93e0-4b4d9b91249f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.715441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d2051-6071-4dc1-93e0-4b4d9b91249f admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.716107 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d2051-6071-4dc1-93e0-4b4d9b91249f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.718043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d2051-6071-4dc1-93e0-4b4d9b91249f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.744658 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1380/2785] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:59.748764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.759050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b61b51-a1cc-4476-a3ed-00884c1bc338 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.814516 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079f24-dbfa-44f1-a3d9-b73d9ce4e1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.815111 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079f24-dbfa-44f1-a3d9-b73d9ce4e1d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.815383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079f24-dbfa-44f1-a3d9-b73d9ce4e1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.815651 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78079f24-dbfa-44f1-a3d9-b73d9ce4e1d9 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.816269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78079f24-dbfa-44f1-a3d9-b73d9ce4e1d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.818189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78079f24-dbfa-44f1-a3d9-b73d9ce4e1d9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.838775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1406/2786] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:59.842862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d2051-6071-4dc1-93e0-4b4d9b91249f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.852961 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d2051-6071-4dc1-93e0-4b4d9b91249f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.857511 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4b8e0-f2c7-48f0-9f64-48d39288ae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.858128 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b8e0-f2c7-48f0-9f64-48d39288ae9a admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.858409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b8e0-f2c7-48f0-9f64-48d39288ae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.858681 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4b8e0-f2c7-48f0-9f64-48d39288ae9a admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.859346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4b8e0-f2c7-48f0-9f64-48d39288ae9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.860942 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4b8e0-f2c7-48f0-9f64-48d39288ae9a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.870897 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1381/2787] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:59.875291 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079f24-dbfa-44f1-a3d9-b73d9ce4e1d9 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.884914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78079f24-dbfa-44f1-a3d9-b73d9ce4e1d9 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.890203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c4957-8734-4b0e-81d8-1bd37d1ebc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.890809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c4957-8734-4b0e-81d8-1bd37d1ebc46 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.891094 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c4957-8734-4b0e-81d8-1bd37d1ebc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.891363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c4957-8734-4b0e-81d8-1bd37d1ebc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.891997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c4957-8734-4b0e-81d8-1bd37d1ebc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.893575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c4957-8734-4b0e-81d8-1bd37d1ebc46 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.906949 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1407/2788] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:59.908463 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4b8e0-f2c7-48f0-9f64-48d39288ae9a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.918441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4b8e0-f2c7-48f0-9f64-48d39288ae9a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.923125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.923773 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.924058 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.924340 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] PATH_INFO: `/v3/projects/d14ffba2cefc4248bc0fa095c4662601/users/0b5c1ed6b8e04486b717424df36d1599/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.939742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14ffba2cefc4248bc0fa095c4662601, user_id=0b5c1ed6b8e04486b717424df36d1599, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.942456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.958545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1382/2789] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:59 2022] PUT /identity/v3/projects/d14ffba2cefc4248bc0fa095c4662601/users/0b5c1ed6b8e04486b717424df36d1599/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:59.968808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa737-c144-4d99-aa2e-241f3404adcc None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.969100 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa737-c144-4d99-aa2e-241f3404adcc None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.969375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeaa737-c144-4d99-aa2e-241f3404adcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.000124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-6aeaa737-c144-4d99-aa2e-241f3404adcc None None] MFA Rules not processed for user `0b5c1ed6b8e04486b717424df36d1599`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:00.006410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6aeaa737-c144-4d99-aa2e-241f3404adcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:00.032139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:00.085481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1408/2790] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:00.093045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.094876 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:00.157575 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f3dcf-d807-49e9-b9ab-966410af00e8 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.161360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.161992 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.162273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.162546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.172183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.173660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.183727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1383/2791] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:01.575797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2365-6fe1-4567-842f-14aabf679aa0 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:01.576395 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2365-6fe1-4567-842f-14aabf679aa0 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:01.576508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b2365-6fe1-4567-842f-14aabf679aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:01.595613 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ea1b2365-6fe1-4567-842f-14aabf679aa0 None None] MFA Rules not processed for user `0b5c1ed6b8e04486b717424df36d1599`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:01.596835 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ea1b2365-6fe1-4567-842f-14aabf679aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:01.604878 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1409/2792] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:27:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:10.178739 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:10.512085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:10.513526 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:10.577201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c559da8-882d-4406-90b3-03a4ce3c510e service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:10.581723 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:10.582341 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:10.582627 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:10.582904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] PATH_INFO: `/v3/users/228e2723bd124f48a6aa3f19ccb349ef` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:10.584037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=228e2723bd124f48a6aa3f19ccb349ef)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:10.585976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:10.631473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228e2723bd124f48a6aa3f19ccb349ef'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:10.639660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228e2723bd124f48a6aa3f19ccb349ef'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:10.640322 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228e2723bd124f48a6aa3f19ccb349ef'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:10.653122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228e2723bd124f48a6aa3f19ccb349ef'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:10.659968 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228e2723bd124f48a6aa3f19ccb349ef'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:10.665750 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1384/2793] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:10 2022] DELETE /identity/v3/users/228e2723bd124f48a6aa3f19ccb349ef => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:10.670743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c672e72-378e-4bcb-a76e-95adbbe2c89e None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:10.671259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c672e72-378e-4bcb-a76e-95adbbe2c89e None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:10.671366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c672e72-378e-4bcb-a76e-95adbbe2c89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:10.690379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-8c672e72-378e-4bcb-a76e-95adbbe2c89e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:10.691580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8c672e72-378e-4bcb-a76e-95adbbe2c89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:10.751943 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1410/2794] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:10.761548 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:10.762973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:10.819482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3958ef73-32f3-4800-8701-fdd770987683 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:10.823068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67089b81-98cd-4cef-b4e6-35f0db8c864c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:10.823659 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089b81-98cd-4cef-b4e6-35f0db8c864c service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:10.823935 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089b81-98cd-4cef-b4e6-35f0db8c864c service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:10.824204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67089b81-98cd-4cef-b4e6-35f0db8c864c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:10.832785 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67089b81-98cd-4cef-b4e6-35f0db8c864c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:10.834253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67089b81-98cd-4cef-b4e6-35f0db8c864c service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:10.843863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1385/2795] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:11.111099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c672e72-378e-4bcb-a76e-95adbbe2c89e None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:11.112559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8c672e72-378e-4bcb-a76e-95adbbe2c89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:11.125020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c672e72-378e-4bcb-a76e-95adbbe2c89e None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:11.129735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:11.130367 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:11.130638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:11.130902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] PATH_INFO: `/v3/projects/9639394acb914c00b034ba3983bee565` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:11.132101 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9639394acb914c00b034ba3983bee565)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:11.134079 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:11.183222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9639394acb914c00b034ba3983bee565', 'request_id': 'req-fde21566-c765-413e-8888-81e4945d94fd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:11.188837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9639394acb914c00b034ba3983bee565', 'request_id': 'req-fde21566-c765-413e-8888-81e4945d94fd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:11.195170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9639394acb914c00b034ba3983bee565', 'request_id': 'req-fde21566-c765-413e-8888-81e4945d94fd'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:11.209019 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] The token cache is being invalidate because project 9639394acb914c00b034ba3983bee565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:11.209311 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:11.209953 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:11.211451 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1411/2796] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:11 2022] DELETE /identity/v3/projects/9639394acb914c00b034ba3983bee565 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:18.673173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcfbaf-7089-44ef-ae17-40c05e6bd0c7 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.673760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcfbaf-7089-44ef-ae17-40c05e6bd0c7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.673760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcfbaf-7089-44ef-ae17-40c05e6bd0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.703523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:18.723602 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-32bcfbaf-7089-44ef-ae17-40c05e6bd0c7 None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:18.724806 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-32bcfbaf-7089-44ef-ae17-40c05e6bd0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.797024 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1386/2797] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 20:27:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:18.843211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:18.844645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.890843 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:18.937153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde21566-c765-413e-8888-81e4945d94fd admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:18.942318 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7144bd8-b85f-47ee-bd99-8fa073fa29fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:18.942925 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7144bd8-b85f-47ee-bd99-8fa073fa29fa service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.943204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7144bd8-b85f-47ee-bd99-8fa073fa29fa service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.943478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7144bd8-b85f-47ee-bd99-8fa073fa29fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.952108 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7144bd8-b85f-47ee-bd99-8fa073fa29fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:18.953715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7144bd8-b85f-47ee-bd99-8fa073fa29fa service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:18.961689 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1412/2798] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 20:27:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:27:19.458347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f16e-21ee-44a7-b311-968c0a285696 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.458634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f16e-21ee-44a7-b311-968c0a285696 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.458906 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f16e-21ee-44a7-b311-968c0a285696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.477905 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f8e0f16e-21ee-44a7-b311-968c0a285696 None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:19.479766 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f8e0f16e-21ee-44a7-b311-968c0a285696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.486607 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1387/2799] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:19.500105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7144bd8-b85f-47ee-bd99-8fa073fa29fa service swift] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.510954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7144bd8-b85f-47ee-bd99-8fa073fa29fa service swift] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.516003 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e39f-ab70-4bc0-8937-e0d740b36caa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.516589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e39f-ab70-4bc0-8937-e0d740b36caa service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.516864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e39f-ab70-4bc0-8937-e0d740b36caa service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.517130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7e39f-ab70-4bc0-8937-e0d740b36caa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.525031 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7e39f-ab70-4bc0-8937-e0d740b36caa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.526628 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7e39f-ab70-4bc0-8937-e0d740b36caa service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.534422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1413/2800] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 20:27:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:27:19.765797 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aef82-cb05-4b3d-81e8-9655389f2cc7 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.766266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aef82-cb05-4b3d-81e8-9655389f2cc7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.766385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aef82-cb05-4b3d-81e8-9655389f2cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.785076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-5c5aef82-cb05-4b3d-81e8-9655389f2cc7 None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:19.786315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5c5aef82-cb05-4b3d-81e8-9655389f2cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.793462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1388/2801] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:19.806967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e39f-ab70-4bc0-8937-e0d740b36caa service swift] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.817727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7e39f-ab70-4bc0-8937-e0d740b36caa service swift] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.822790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ee557-4657-4dd9-bce9-a142d2c22435 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.823385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ee557-4657-4dd9-bce9-a142d2c22435 service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.823666 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ee557-4657-4dd9-bce9-a142d2c22435 service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.823938 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ee557-4657-4dd9-bce9-a142d2c22435 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.831896 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ee557-4657-4dd9-bce9-a142d2c22435 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.833486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ee557-4657-4dd9-bce9-a142d2c22435 service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.841407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1414/2802] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 20:27:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:27:20.162269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca8e9-973a-4fa9-8fca-cb96b0ccc858 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.162560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca8e9-973a-4fa9-8fca-cb96b0ccc858 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.162834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bca8e9-973a-4fa9-8fca-cb96b0ccc858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.182246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f7bca8e9-973a-4fa9-8fca-cb96b0ccc858 None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:20.183440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f7bca8e9-973a-4fa9-8fca-cb96b0ccc858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:20.190516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1389/2803] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 13 20:27:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:20.203011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ee557-4657-4dd9-bce9-a142d2c22435 service swift] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.213536 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ee557-4657-4dd9-bce9-a142d2c22435 service swift] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.219207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddd2de-bf62-466a-b674-21542d1955bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.219821 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddd2de-bf62-466a-b674-21542d1955bb service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.220097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddd2de-bf62-466a-b674-21542d1955bb service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.220365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddd2de-bf62-466a-b674-21542d1955bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.228210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddd2de-bf62-466a-b674-21542d1955bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.229946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddd2de-bf62-466a-b674-21542d1955bb service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.237782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1415/2804] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 20:27:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:27:22.310582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca8e9-973a-4fa9-8fca-cb96b0ccc858 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:22.312464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f7bca8e9-973a-4fa9-8fca-cb96b0ccc858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:22.376506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bca8e9-973a-4fa9-8fca-cb96b0ccc858 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:22.381007 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:22.381650 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:22.381958 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:22.382231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] PATH_INFO: `/v3/users/0b5c1ed6b8e04486b717424df36d1599` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:22.383358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5c1ed6b8e04486b717424df36d1599)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:22.385292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:22.433527 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5c1ed6b8e04486b717424df36d1599'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.442327 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5c1ed6b8e04486b717424df36d1599'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.443012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5c1ed6b8e04486b717424df36d1599'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.457098 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5c1ed6b8e04486b717424df36d1599'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.463727 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:22.467347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5c1ed6b8e04486b717424df36d1599'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.475558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1390/2805] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:22 2022] DELETE /identity/v3/users/0b5c1ed6b8e04486b717424df36d1599 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:22.480127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760cea7-65c5-4bd9-bca1-5a00f5d47fbb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:22.480605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760cea7-65c5-4bd9-bca1-5a00f5d47fbb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:22.480742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760cea7-65c5-4bd9-bca1-5a00f5d47fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:22.501128 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-0760cea7-65c5-4bd9-bca1-5a00f5d47fbb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:22.502467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0760cea7-65c5-4bd9-bca1-5a00f5d47fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:22.563648 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1416/2806] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:22.573110 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:22.574741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:22.632257 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d164ee-ef79-42ed-aa53-67c25a876256 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:22.636067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:22.636660 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:22.636938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:22.637209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:22.645950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:22.647366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:22.656492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1391/2807] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:22.826424 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760cea7-65c5-4bd9-bca1-5a00f5d47fbb None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:22.827946 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0760cea7-65c5-4bd9-bca1-5a00f5d47fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:22.840428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760cea7-65c5-4bd9-bca1-5a00f5d47fbb None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:22.846921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:22.847556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:22.847833 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:22.848130 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] PATH_INFO: `/v3/projects/d14ffba2cefc4248bc0fa095c4662601` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:22.849309 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14ffba2cefc4248bc0fa095c4662601)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:22.852013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:22.900364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14ffba2cefc4248bc0fa095c4662601', 'request_id': 'req-4f032710-f808-4faf-839d-4d129ced35ee'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.906374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14ffba2cefc4248bc0fa095c4662601', 'request_id': 'req-4f032710-f808-4faf-839d-4d129ced35ee'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.912482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14ffba2cefc4248bc0fa095c4662601', 'request_id': 'req-4f032710-f808-4faf-839d-4d129ced35ee'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.916404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:22.929858 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] The token cache is being invalidate because project d14ffba2cefc4248bc0fa095c4662601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:22.930317 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.931189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:22.932663 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1417/2808] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:22 2022] DELETE /identity/v3/projects/d14ffba2cefc4248bc0fa095c4662601 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:32.652433 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=29 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:32.930993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=2 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:36.658490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:36.659863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:36.778496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde08fa-fe4a-425b-bc7e-f6a6137381ac service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:36.836185 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:36.836787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:36.837065 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:36.837343 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:36.839085 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:36.849943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:36.851516 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:36.861704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1392/2809] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:46.856892 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=80 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:29:59.042556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:29:59.043991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:59.121344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f032710-f808-4faf-839d-4d129ced35ee admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:29:59.125057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:29:59.125758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:29:59.126042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:29:59.126328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:29:59.128077 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:29:59.196659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:29:59.198123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:29:59.207114 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1418/2810] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:29:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:30:09.202913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=62 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:30:28.160971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:30:28.162528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:30:28.243976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1931d64-0a17-4f9e-b94c-91b023eae1de service placement] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:30:28.247555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:30:28.248749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:30:28.249028 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:30:28.249296 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:30:28.250994 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:30:28.260056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:30:28.261556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:30:28.269364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1393/2811] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:30:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:30:38.265977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=39 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:02.762281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:02.763772 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:02.871207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb81526-a983-4997-80ee-845e52623cf3 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:02.877279 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:02.878029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:02.878575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:02.878911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:02.881276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:02.895584 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:02.897196 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:02.910704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1419/2812] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:04.406397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:04.407868 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:04.418324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a825e6-f6ce-4caa-ac39-5750cf481fff service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:04.422445 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:04.423036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:04.423315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:04.423587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:04.425234 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:04.438716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:04.440152 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:04.450944 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1394/2813] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:06.743348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:06.752623 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145df11-7559-4450-b8b4-7ed006c67783 service nova] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:06.756911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:06.757641 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:06.757942 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:06.758210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:06.759932 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:06.822678 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:06.824095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:06.834339 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1420/2814] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:09.875127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:09.876536 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:09.891122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf59be7-9f72-44cf-a88f-9fe601341c17 service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:09.896211 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:09.896823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:09.897099 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:09.897372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:09.899201 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:09.911068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:09.912764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:09.927125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1395/2815] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:09.943649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:09.946030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:09.962977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55929a-9ddb-4ab1-a6fc-7eda997bfdb6 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:09.968236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:09.968890 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:09.969165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:09.969434 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] PATH_INFO: `/v3/users/a0e295fda2af4157b196c6130aa2d6d2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:09.970582 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e295fda2af4157b196c6130aa2d6d2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:09.972474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:10.022704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e295fda2af4157b196c6130aa2d6d2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.032351 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e295fda2af4157b196c6130aa2d6d2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.033020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e295fda2af4157b196c6130aa2d6d2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.037502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:10.047124 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e295fda2af4157b196c6130aa2d6d2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.053945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-0164d537-2739-43ef-8bc3-5392171b9bd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e295fda2af4157b196c6130aa2d6d2'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.059811 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1421/2816] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:31:09 2022] DELETE /identity/v3/users/a0e295fda2af4157b196c6130aa2d6d2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:10.226213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:10.246269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3792a4-d544-4781-bcb7-7d6002ac58d0 service placement] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:10.303744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:10.304292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.304572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.304844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] PATH_INFO: `/v3/projects/6991f30f2c094499b73579677b12abae` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.305976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6991f30f2c094499b73579677b12abae)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:10.307898 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:10.355038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6991f30f2c094499b73579677b12abae', 'request_id': 'req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.361859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6991f30f2c094499b73579677b12abae', 'request_id': 'req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.367991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6991f30f2c094499b73579677b12abae', 'request_id': 'req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.381719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] The token cache is being invalidate because project 6991f30f2c094499b73579677b12abae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:31:10.382071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.382755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-bb2f5938-8c25-40b2-8a51-b38e9cad0a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:10.384690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1396/2817] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:31:10 2022] DELETE /identity/v3/projects/6991f30f2c094499b73579677b12abae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:10.390718 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9049f-9da9-409b-8a8c-638ee8bf1a0f None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.391069 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9049f-9da9-409b-8a8c-638ee8bf1a0f None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.391324 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9049f-9da9-409b-8a8c-638ee8bf1a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.411013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-63c9049f-9da9-409b-8a8c-638ee8bf1a0f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:10.412211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-63c9049f-9da9-409b-8a8c-638ee8bf1a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:10.469993 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1422/2818] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:10.477748 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89e294-7528-4ded-9533-1e005bb76700 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.478036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89e294-7528-4ded-9533-1e005bb76700 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.478310 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89e294-7528-4ded-9533-1e005bb76700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.498180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-cf89e294-7528-4ded-9533-1e005bb76700 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:10.499981 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-cf89e294-7528-4ded-9533-1e005bb76700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:10.507689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1397/2819] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:10.511871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9049f-9da9-409b-8a8c-638ee8bf1a0f None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:10.521404 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9049f-9da9-409b-8a8c-638ee8bf1a0f None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:10.525988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:10.526577 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.526855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.527131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.527780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:31:10.528081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:10.529691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:10.540976 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1423/2820] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:31:10.544893 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89e294-7528-4ded-9533-1e005bb76700 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:10.554721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89e294-7528-4ded-9533-1e005bb76700 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:10.559347 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fef64d-057b-4ce4-b3bf-6dd83d912356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:10.559932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fef64d-057b-4ce4-b3bf-6dd83d912356 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.560209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fef64d-057b-4ce4-b3bf-6dd83d912356 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.560478 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fef64d-057b-4ce4-b3bf-6dd83d912356 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.561124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fef64d-057b-4ce4-b3bf-6dd83d912356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:10.563678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fef64d-057b-4ce4-b3bf-6dd83d912356 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:10.589852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1398/2821] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:10 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:10.594426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:10.604265 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fd512-ffe1-45c5-9267-67545e7b30ba admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:10.659806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774342c-9d15-4878-a0d9-20c534a67aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:10.660377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774342c-9d15-4878-a0d9-20c534a67aed admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.660658 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774342c-9d15-4878-a0d9-20c534a67aed admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.660933 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774342c-9d15-4878-a0d9-20c534a67aed admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.661570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774342c-9d15-4878-a0d9-20c534a67aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:10.663726 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774342c-9d15-4878-a0d9-20c534a67aed admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:10.683980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1424/2822] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:31:10 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:10.688238 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fef64d-057b-4ce4-b3bf-6dd83d912356 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:10.698519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fef64d-057b-4ce4-b3bf-6dd83d912356 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:10.703057 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed8bd3-d984-4ad5-a997-5b2a311616a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:10.703647 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed8bd3-d984-4ad5-a997-5b2a311616a1 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.703927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed8bd3-d984-4ad5-a997-5b2a311616a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.704200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed8bd3-d984-4ad5-a997-5b2a311616a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.704837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed8bd3-d984-4ad5-a997-5b2a311616a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:10.706473 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed8bd3-d984-4ad5-a997-5b2a311616a1 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:10.716182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1399/2823] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:10.720546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774342c-9d15-4878-a0d9-20c534a67aed admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:10.730407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774342c-9d15-4878-a0d9-20c534a67aed admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:10.735841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f5fce-483f-43e6-b250-d14463f7b29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:10.736422 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5fce-483f-43e6-b250-d14463f7b29d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.736697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5fce-483f-43e6-b250-d14463f7b29d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.736968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5fce-483f-43e6-b250-d14463f7b29d admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.737619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5fce-483f-43e6-b250-d14463f7b29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:10.739198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5fce-483f-43e6-b250-d14463f7b29d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:10.749716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1425/2824] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:10.753449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed8bd3-d984-4ad5-a997-5b2a311616a1 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:10.763586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed8bd3-d984-4ad5-a997-5b2a311616a1 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:10.768132 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9b1c5-25db-4024-969c-479b2e5c6f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:10.768759 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9b1c5-25db-4024-969c-479b2e5c6f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.769035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9b1c5-25db-4024-969c-479b2e5c6f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.769305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9b1c5-25db-4024-969c-479b2e5c6f09 admin admin] PATH_INFO: `/v3/projects/0b273b216f9344889691c2dfe7d5f942/users/9a5c4586919c4e5fb961c691c8375a69/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.784193 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9b1c5-25db-4024-969c-479b2e5c6f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b273b216f9344889691c2dfe7d5f942, user_id=9a5c4586919c4e5fb961c691c8375a69, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:10.786943 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea9b1c5-25db-4024-969c-479b2e5c6f09 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:10.795029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:10.800971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1400/2825] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:10 2022] PUT /identity/v3/projects/0b273b216f9344889691c2dfe7d5f942/users/9a5c4586919c4e5fb961c691c8375a69/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:10.812741 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f35c-f78f-4c62-914b-8f2d6ec6ba66 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.813052 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f35c-f78f-4c62-914b-8f2d6ec6ba66 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.813328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7f35c-f78f-4c62-914b-8f2d6ec6ba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.841043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-5dc7f35c-f78f-4c62-914b-8f2d6ec6ba66 None None] MFA Rules not processed for user `9a5c4586919c4e5fb961c691c8375a69`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:10.842281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5dc7f35c-f78f-4c62-914b-8f2d6ec6ba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:10.871914 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:10.906944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1426/2826] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:10.911384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9b1c5-25db-4024-969c-479b2e5c6f09 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:10.972688 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9b1c5-25db-4024-969c-479b2e5c6f09 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:10.977265 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e171c98-0048-419c-a8eb-9f3be8c20bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:10.977899 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e171c98-0048-419c-a8eb-9f3be8c20bf6 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.978179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e171c98-0048-419c-a8eb-9f3be8c20bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.978456 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e171c98-0048-419c-a8eb-9f3be8c20bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.979093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e171c98-0048-419c-a8eb-9f3be8c20bf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:10.981029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e171c98-0048-419c-a8eb-9f3be8c20bf6 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.006525 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1401/2827] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:10 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:11.010743 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7f35c-f78f-4c62-914b-8f2d6ec6ba66 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:11.022250 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7f35c-f78f-4c62-914b-8f2d6ec6ba66 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:11.081486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffc56d-5f8e-4a8c-bc1b-f37d12f78779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:11.082143 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffc56d-5f8e-4a8c-bc1b-f37d12f78779 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.082426 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffc56d-5f8e-4a8c-bc1b-f37d12f78779 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.082699 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffc56d-5f8e-4a8c-bc1b-f37d12f78779 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.083341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffc56d-5f8e-4a8c-bc1b-f37d12f78779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:11.085274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffc56d-5f8e-4a8c-bc1b-f37d12f78779 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.107817 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1427/2828] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:31:11 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:11.112354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e171c98-0048-419c-a8eb-9f3be8c20bf6 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:11.122556 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e171c98-0048-419c-a8eb-9f3be8c20bf6 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:11.127707 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de726c4d-dc59-4bab-bec2-53dad3e54ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:11.128195 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726c4d-dc59-4bab-bec2-53dad3e54ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.128480 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726c4d-dc59-4bab-bec2-53dad3e54ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.128754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de726c4d-dc59-4bab-bec2-53dad3e54ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.129475 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de726c4d-dc59-4bab-bec2-53dad3e54ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:11.131077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de726c4d-dc59-4bab-bec2-53dad3e54ea3 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.142665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1402/2829] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:11.147007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffc56d-5f8e-4a8c-bc1b-f37d12f78779 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:11.157696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffc56d-5f8e-4a8c-bc1b-f37d12f78779 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:11.162440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b481e0-ad73-4f0f-99b8-385a3283b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:11.163185 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b481e0-ad73-4f0f-99b8-385a3283b457 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.163461 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b481e0-ad73-4f0f-99b8-385a3283b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.163809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b481e0-ad73-4f0f-99b8-385a3283b457 admin admin] PATH_INFO: `/v3/projects/35c29c507d884bbc93db463f3149f464/users/227202e646574130b00d6b7bbed273da/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.178227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b481e0-ad73-4f0f-99b8-385a3283b457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c29c507d884bbc93db463f3149f464, user_id=227202e646574130b00d6b7bbed273da, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:11.180882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b481e0-ad73-4f0f-99b8-385a3283b457 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.193268 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1428/2830] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:11 2022] PUT /identity/v3/projects/35c29c507d884bbc93db463f3149f464/users/227202e646574130b00d6b7bbed273da/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:11.197288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de726c4d-dc59-4bab-bec2-53dad3e54ea3 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:11.207546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de726c4d-dc59-4bab-bec2-53dad3e54ea3 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:11.263254 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9a584-b0f4-4d71-878a-b3c94853c382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:11.263848 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9a584-b0f4-4d71-878a-b3c94853c382 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.264129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9a584-b0f4-4d71-878a-b3c94853c382 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.264400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9a584-b0f4-4d71-878a-b3c94853c382 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.265713 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9a584-b0f4-4d71-878a-b3c94853c382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:11.267303 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9a584-b0f4-4d71-878a-b3c94853c382 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.276716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1403/2831] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:11.281601 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b481e0-ad73-4f0f-99b8-385a3283b457 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:11.291849 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b481e0-ad73-4f0f-99b8-385a3283b457 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:11.296537 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:11.297190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.297465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.297780 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] PATH_INFO: `/v3/projects/35c29c507d884bbc93db463f3149f464/users/227202e646574130b00d6b7bbed273da/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.299691 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35c29c507d884bbc93db463f3149f464, user_id=227202e646574130b00d6b7bbed273da, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:11.302372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.313195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1429/2832] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:11 2022] PUT /identity/v3/projects/35c29c507d884bbc93db463f3149f464/users/227202e646574130b00d6b7bbed273da/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:11.324306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b65ad-8643-4a58-85df-3b54addca1bb None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.324583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b65ad-8643-4a58-85df-3b54addca1bb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.325078 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b65ad-8643-4a58-85df-3b54addca1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.352315 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8f6b65ad-8643-4a58-85df-3b54addca1bb None None] MFA Rules not processed for user `227202e646574130b00d6b7bbed273da`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:11.353529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8f6b65ad-8643-4a58-85df-3b54addca1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:11.415740 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:11.425336 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1404/2833] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:11.436941 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:11.438620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:11.505911 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e026271a-291b-499d-a04b-e557e47d2d89 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:11.510341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9cb02-fe2b-44cb-95d5-364dcdc934b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:11.510954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9cb02-fe2b-44cb-95d5-364dcdc934b3 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.511231 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9cb02-fe2b-44cb-95d5-364dcdc934b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.511504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9cb02-fe2b-44cb-95d5-364dcdc934b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.522109 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9cb02-fe2b-44cb-95d5-364dcdc934b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:11.523552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9cb02-fe2b-44cb-95d5-364dcdc934b3 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.532987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1430/2834] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:11.606278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b65ad-8643-4a58-85df-3b54addca1bb None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:11.616298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b65ad-8643-4a58-85df-3b54addca1bb None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:11.619927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a8569-4296-4d7a-9267-4657c8d006e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:11.620537 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8569-4296-4d7a-9267-4657c8d006e6 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.620805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8569-4296-4d7a-9267-4657c8d006e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.621080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a8569-4296-4d7a-9267-4657c8d006e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.676494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a8569-4296-4d7a-9267-4657c8d006e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:11.677973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a8569-4296-4d7a-9267-4657c8d006e6 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.687242 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1405/2835] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:21.528815 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:21.682731 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=30 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:53.049995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9cb02-fe2b-44cb-95d5-364dcdc934b3 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:53.112349 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9cb02-fe2b-44cb-95d5-364dcdc934b3 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:53.116963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:53.117659 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:53.117940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:53.118241 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] PATH_INFO: `/v3/users/9a5c4586919c4e5fb961c691c8375a69` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:53.119353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5c4586919c4e5fb961c691c8375a69)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:53.121269 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:53.166330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5c4586919c4e5fb961c691c8375a69'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.173841 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5c4586919c4e5fb961c691c8375a69'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.174480 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5c4586919c4e5fb961c691c8375a69'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.188538 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5c4586919c4e5fb961c691c8375a69'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.195348 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5c4586919c4e5fb961c691c8375a69'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.201336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1431/2836] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:31:53 2022] DELETE /identity/v3/users/9a5c4586919c4e5fb961c691c8375a69 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:53.206585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71454-be59-45ae-942a-5289a8c406b5 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:53.207068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71454-be59-45ae-942a-5289a8c406b5 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:53.207179 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b71454-be59-45ae-942a-5289a8c406b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:53.234400 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-11b71454-be59-45ae-942a-5289a8c406b5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:53.235489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-11b71454-be59-45ae-942a-5289a8c406b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:53.300778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1406/2837] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:53.310366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:53.312337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:53.368347 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72251ee-9c36-46c9-bb00-c084ebc9861e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:53.371963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664a4da-c869-474d-80b9-3b16933d512f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:53.372560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664a4da-c869-474d-80b9-3b16933d512f service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:53.372839 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664a4da-c869-474d-80b9-3b16933d512f service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:53.373113 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664a4da-c869-474d-80b9-3b16933d512f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:53.382508 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664a4da-c869-474d-80b9-3b16933d512f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:53.383918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664a4da-c869-474d-80b9-3b16933d512f service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:53.393794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1432/2838] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:53.662056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71454-be59-45ae-942a-5289a8c406b5 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:53.663594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-11b71454-be59-45ae-942a-5289a8c406b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:53.677594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b71454-be59-45ae-942a-5289a8c406b5 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:53.682372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:53.682997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:53.683279 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:53.683553 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] PATH_INFO: `/v3/projects/0b273b216f9344889691c2dfe7d5f942` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:53.684694 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b273b216f9344889691c2dfe7d5f942)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:53.686685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:53.733000 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b273b216f9344889691c2dfe7d5f942', 'request_id': 'req-126d04ba-a65d-4627-bfc7-be8979353997'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.739450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b273b216f9344889691c2dfe7d5f942', 'request_id': 'req-126d04ba-a65d-4627-bfc7-be8979353997'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.745562 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b273b216f9344889691c2dfe7d5f942', 'request_id': 'req-126d04ba-a65d-4627-bfc7-be8979353997'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.758266 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] The token cache is being invalidate because project 0b273b216f9344889691c2dfe7d5f942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:31:53.758563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.759200 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.760709 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1407/2839] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:31:53 2022] DELETE /identity/v3/projects/0b273b216f9344889691c2dfe7d5f942 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:53.764732 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664a4da-c869-474d-80b9-3b16933d512f service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:53.774465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664a4da-c869-474d-80b9-3b16933d512f service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:53.791977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:53.832129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:53.832753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:53.833025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:53.833301 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] PATH_INFO: `/v3/users/227202e646574130b00d6b7bbed273da` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:53.834440 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=227202e646574130b00d6b7bbed273da)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:53.836338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:53.881730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '227202e646574130b00d6b7bbed273da'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.889320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '227202e646574130b00d6b7bbed273da'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.889967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '227202e646574130b00d6b7bbed273da'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.901977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '227202e646574130b00d6b7bbed273da'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.908710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-b43bc5c9-17f2-46fd-90c1-cc02f8eea6fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '227202e646574130b00d6b7bbed273da'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:53.914504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1433/2840] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:31:53 2022] DELETE /identity/v3/users/227202e646574130b00d6b7bbed273da => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:54.174590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.176045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:54.188971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d04ba-a65d-4627-bfc7-be8979353997 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:54.246004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:54.246623 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.246904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.247178 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] PATH_INFO: `/v3/projects/35c29c507d884bbc93db463f3149f464` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.248285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35c29c507d884bbc93db463f3149f464)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:54.250244 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:54.279119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:54.297444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35c29c507d884bbc93db463f3149f464', 'request_id': 'req-d9f08bf9-1f00-4323-a389-8c7cde96c656'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:54.303021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35c29c507d884bbc93db463f3149f464', 'request_id': 'req-d9f08bf9-1f00-4323-a389-8c7cde96c656'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:54.309034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35c29c507d884bbc93db463f3149f464', 'request_id': 'req-d9f08bf9-1f00-4323-a389-8c7cde96c656'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:54.320993 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] The token cache is being invalidate because project 35c29c507d884bbc93db463f3149f464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:31:54.321285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:54.321932 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d9f08bf9-1f00-4323-a389-8c7cde96c656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:54.323438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1408/2841] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:31:54 2022] DELETE /identity/v3/projects/35c29c507d884bbc93db463f3149f464 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:54.328281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7777-4007-4e95-884c-7cf7ad752ed5 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.328745 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7777-4007-4e95-884c-7cf7ad752ed5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.328848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae7777-4007-4e95-884c-7cf7ad752ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.348645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-91ae7777-4007-4e95-884c-7cf7ad752ed5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:54.349870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-91ae7777-4007-4e95-884c-7cf7ad752ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:54.408531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1434/2842] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:54.416081 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a01b12-1c1b-4e9f-baef-9cc549172f0f None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.416373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a01b12-1c1b-4e9f-baef-9cc549172f0f None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.416654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a01b12-1c1b-4e9f-baef-9cc549172f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.435781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-47a01b12-1c1b-4e9f-baef-9cc549172f0f None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:54.436995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-47a01b12-1c1b-4e9f-baef-9cc549172f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:54.444761 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1409/2843] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:54.448757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae7777-4007-4e95-884c-7cf7ad752ed5 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.458930 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae7777-4007-4e95-884c-7cf7ad752ed5 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:54.463474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:54.464067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.464343 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.464616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.465253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:31:54.465550 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:54.467170 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:54.479195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1435/2844] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:31:54.483323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a01b12-1c1b-4e9f-baef-9cc549172f0f None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.492794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a01b12-1c1b-4e9f-baef-9cc549172f0f None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:54.497360 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff39b54-2008-4dad-ab56-1a6f9a5e6d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:54.498004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff39b54-2008-4dad-ab56-1a6f9a5e6d20 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.498289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff39b54-2008-4dad-ab56-1a6f9a5e6d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.498558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff39b54-2008-4dad-ab56-1a6f9a5e6d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.499219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff39b54-2008-4dad-ab56-1a6f9a5e6d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:54.501126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff39b54-2008-4dad-ab56-1a6f9a5e6d20 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:54.527581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1410/2845] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:54 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:54.532528 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.542256 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5737a9c4-ace1-45c4-9772-f9816ab2869e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:54.567210 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:54.598482 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a866449-5306-4481-a70c-36011e80b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:54.599085 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a866449-5306-4481-a70c-36011e80b593 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.599353 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a866449-5306-4481-a70c-36011e80b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.599625 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a866449-5306-4481-a70c-36011e80b593 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.600247 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a866449-5306-4481-a70c-36011e80b593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:54.602192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a866449-5306-4481-a70c-36011e80b593 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:54.622491 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1436/2846] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:31:54 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:54.626523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff39b54-2008-4dad-ab56-1a6f9a5e6d20 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.636662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff39b54-2008-4dad-ab56-1a6f9a5e6d20 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:54.641834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d4679-436b-48fe-8f9e-d34a6ad5608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:54.642421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d4679-436b-48fe-8f9e-d34a6ad5608c admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.642699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d4679-436b-48fe-8f9e-d34a6ad5608c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.642967 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d4679-436b-48fe-8f9e-d34a6ad5608c admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.643595 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d4679-436b-48fe-8f9e-d34a6ad5608c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:54.645177 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d4679-436b-48fe-8f9e-d34a6ad5608c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:54.654778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1411/2847] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:54.659215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a866449-5306-4481-a70c-36011e80b593 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.669132 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a866449-5306-4481-a70c-36011e80b593 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:54.673860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe14328-b2d3-42f8-aecb-6a77e640bd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:54.674454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14328-b2d3-42f8-aecb-6a77e640bd51 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.674733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14328-b2d3-42f8-aecb-6a77e640bd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.675006 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe14328-b2d3-42f8-aecb-6a77e640bd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.675645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe14328-b2d3-42f8-aecb-6a77e640bd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:54.677222 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe14328-b2d3-42f8-aecb-6a77e640bd51 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:54.687656 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1437/2848] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:54.691514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d4679-436b-48fe-8f9e-d34a6ad5608c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.701237 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d4679-436b-48fe-8f9e-d34a6ad5608c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:54.706489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73177e-f5b5-4402-84c4-f7ea7ec55c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:54.707126 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73177e-f5b5-4402-84c4-f7ea7ec55c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.707406 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73177e-f5b5-4402-84c4-f7ea7ec55c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.707678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b73177e-f5b5-4402-84c4-f7ea7ec55c1e admin admin] PATH_INFO: `/v3/projects/6ccea5f749a946fb8ef2ec2909f84834/users/f94519efa7884e0eb61713233375e9d2/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.722567 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73177e-f5b5-4402-84c4-f7ea7ec55c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ccea5f749a946fb8ef2ec2909f84834, user_id=f94519efa7884e0eb61713233375e9d2, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:54.725221 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b73177e-f5b5-4402-84c4-f7ea7ec55c1e admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:54.738389 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1412/2849] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:54 2022] PUT /identity/v3/projects/6ccea5f749a946fb8ef2ec2909f84834/users/f94519efa7884e0eb61713233375e9d2/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:54.748213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3675dd1-de14-4732-8f59-24afa02db727 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.748496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3675dd1-de14-4732-8f59-24afa02db727 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.748768 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3675dd1-de14-4732-8f59-24afa02db727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.778161 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a3675dd1-de14-4732-8f59-24afa02db727 None None] MFA Rules not processed for user `f94519efa7884e0eb61713233375e9d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:54.779363 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a3675dd1-de14-4732-8f59-24afa02db727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:54.842095 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1438/2850] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:54.846514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73177e-f5b5-4402-84c4-f7ea7ec55c1e admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.907166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b73177e-f5b5-4402-84c4-f7ea7ec55c1e admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:54.911767 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298baaf-1c14-4dbf-acb9-73849af8488c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:54.912365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298baaf-1c14-4dbf-acb9-73849af8488c admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:54.912646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298baaf-1c14-4dbf-acb9-73849af8488c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:54.912918 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298baaf-1c14-4dbf-acb9-73849af8488c admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:54.913552 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298baaf-1c14-4dbf-acb9-73849af8488c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:54.915487 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d298baaf-1c14-4dbf-acb9-73849af8488c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:54.942006 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1413/2851] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:54 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:54.946177 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3675dd1-de14-4732-8f59-24afa02db727 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:54.957253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3675dd1-de14-4732-8f59-24afa02db727 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:55.012851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bada9-1912-4247-a3dd-591bddcbff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:55.013466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bada9-1912-4247-a3dd-591bddcbff01 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:55.013776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bada9-1912-4247-a3dd-591bddcbff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:55.014049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bada9-1912-4247-a3dd-591bddcbff01 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:55.014720 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bada9-1912-4247-a3dd-591bddcbff01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:55.017142 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255bada9-1912-4247-a3dd-591bddcbff01 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:55.042159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1439/2852] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:31:54 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:55.048287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298baaf-1c14-4dbf-acb9-73849af8488c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:55.058705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d298baaf-1c14-4dbf-acb9-73849af8488c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:55.063841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbc5ee-b518-4620-8301-5601bf8d6c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:55.064437 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbc5ee-b518-4620-8301-5601bf8d6c4a admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:55.064721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbc5ee-b518-4620-8301-5601bf8d6c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:55.064997 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbc5ee-b518-4620-8301-5601bf8d6c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:55.065663 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdbc5ee-b518-4620-8301-5601bf8d6c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:55.067291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdbc5ee-b518-4620-8301-5601bf8d6c4a admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:55.078253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1414/2853] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:55.082416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bada9-1912-4247-a3dd-591bddcbff01 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:55.092091 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bada9-1912-4247-a3dd-591bddcbff01 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:55.096679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a3ee5-0c2d-4189-ad92-cc0eeaf565d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:55.097266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a3ee5-0c2d-4189-ad92-cc0eeaf565d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:55.097542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a3ee5-0c2d-4189-ad92-cc0eeaf565d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:55.097851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a3ee5-0c2d-4189-ad92-cc0eeaf565d9 admin admin] PATH_INFO: `/v3/projects/924d0722ad8a4691b153e2dbf68a710a/users/7a69bb4798d64edab13fa6d2adc11299/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:55.112335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a3ee5-0c2d-4189-ad92-cc0eeaf565d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924d0722ad8a4691b153e2dbf68a710a, user_id=7a69bb4798d64edab13fa6d2adc11299, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:55.115127 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a3ee5-0c2d-4189-ad92-cc0eeaf565d9 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:55.127902 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1440/2854] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:55 2022] PUT /identity/v3/projects/924d0722ad8a4691b153e2dbf68a710a/users/7a69bb4798d64edab13fa6d2adc11299/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:55.131866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbc5ee-b518-4620-8301-5601bf8d6c4a admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:55.142203 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbc5ee-b518-4620-8301-5601bf8d6c4a admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:55.161207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:55.200236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d3a3c-f91e-4db6-98e4-48b431c7da54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:55.200825 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3a3c-f91e-4db6-98e4-48b431c7da54 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:55.201103 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3a3c-f91e-4db6-98e4-48b431c7da54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:55.201372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3a3c-f91e-4db6-98e4-48b431c7da54 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:55.202032 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d3a3c-f91e-4db6-98e4-48b431c7da54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:55.203634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d3a3c-f91e-4db6-98e4-48b431c7da54 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:55.212882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1415/2855] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:31:55.217167 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a3ee5-0c2d-4189-ad92-cc0eeaf565d9 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:55.223071 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:55.228492 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a3ee5-0c2d-4189-ad92-cc0eeaf565d9 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:55.234711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:55.235379 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:55.235661 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:55.235935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] PATH_INFO: `/v3/projects/924d0722ad8a4691b153e2dbf68a710a/users/7a69bb4798d64edab13fa6d2adc11299/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:55.237917 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=924d0722ad8a4691b153e2dbf68a710a, user_id=7a69bb4798d64edab13fa6d2adc11299, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:55.240582 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:55.251197 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1441/2856] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:55 2022] PUT /identity/v3/projects/924d0722ad8a4691b153e2dbf68a710a/users/7a69bb4798d64edab13fa6d2adc11299/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:55.261988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2123c-8ba8-4a65-a2bf-5cba7835e22a None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:55.262489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2123c-8ba8-4a65-a2bf-5cba7835e22a None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:55.262582 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2123c-8ba8-4a65-a2bf-5cba7835e22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:55.289689 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-adc2123c-8ba8-4a65-a2bf-5cba7835e22a None None] MFA Rules not processed for user `7a69bb4798d64edab13fa6d2adc11299`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:55.290954 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-adc2123c-8ba8-4a65-a2bf-5cba7835e22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:55.368364 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1416/2857] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:55 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:55.379797 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:55.381579 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:55.437924 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f071ea-cdf2-4b50-8f95-01aa48f02f5f admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:55.441522 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8494b2d-eb6e-44b4-b5d1-1d7a63c5d749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:55.442151 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8494b2d-eb6e-44b4-b5d1-1d7a63c5d749 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:55.442428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8494b2d-eb6e-44b4-b5d1-1d7a63c5d749 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:55.442697 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8494b2d-eb6e-44b4-b5d1-1d7a63c5d749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:55.451791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8494b2d-eb6e-44b4-b5d1-1d7a63c5d749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:55.453215 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8494b2d-eb6e-44b4-b5d1-1d7a63c5d749 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:55.462383 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1442/2858] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:56.803922 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2123c-8ba8-4a65-a2bf-5cba7835e22a None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:56.864812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2123c-8ba8-4a65-a2bf-5cba7835e22a None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:56.869307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:56.869965 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:56.870369 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:56.870645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] PATH_INFO: `/v3/users/f94519efa7884e0eb61713233375e9d2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:56.871752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f94519efa7884e0eb61713233375e9d2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:56.873714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:56.919474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f94519efa7884e0eb61713233375e9d2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:56.921042 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:56.926628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f94519efa7884e0eb61713233375e9d2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:56.927454 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f94519efa7884e0eb61713233375e9d2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:56.940033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f94519efa7884e0eb61713233375e9d2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:56.946883 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f94519efa7884e0eb61713233375e9d2'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:56.952704 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1417/2859] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:31:56 2022] DELETE /identity/v3/users/f94519efa7884e0eb61713233375e9d2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:56.957806 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ef5a-4b00-4901-a35e-39a4d6f32157 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:56.958263 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ef5a-4b00-4901-a35e-39a4d6f32157 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:56.958372 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ef5a-4b00-4901-a35e-39a4d6f32157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:56.977025 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-29f5ef5a-4b00-4901-a35e-39a4d6f32157 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:56.978273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-29f5ef5a-4b00-4901-a35e-39a4d6f32157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:57.040259 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1443/2860] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:57.050373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:57.051795 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:57.109273 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79931e0-8dc4-4899-80f5-edcb32f136d7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:57.113491 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934eafd-f7f8-4f8a-82d5-9bdf02c2e7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:57.114125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934eafd-f7f8-4f8a-82d5-9bdf02c2e7ad service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:57.114411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934eafd-f7f8-4f8a-82d5-9bdf02c2e7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:57.114687 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934eafd-f7f8-4f8a-82d5-9bdf02c2e7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:57.123586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934eafd-f7f8-4f8a-82d5-9bdf02c2e7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:57.125015 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934eafd-f7f8-4f8a-82d5-9bdf02c2e7ad service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:57.134837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1418/2861] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:57.398096 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5ef5a-4b00-4901-a35e-39a4d6f32157 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:57.399585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-29f5ef5a-4b00-4901-a35e-39a4d6f32157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:57.415029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5ef5a-4b00-4901-a35e-39a4d6f32157 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:57.421013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:57.421909 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:57.422799 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:57.423072 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] PATH_INFO: `/v3/projects/6ccea5f749a946fb8ef2ec2909f84834` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:57.424199 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ccea5f749a946fb8ef2ec2909f84834)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:57.426179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:57.472775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ccea5f749a946fb8ef2ec2909f84834', 'request_id': 'req-24482674-bac6-43ad-91b0-a76aa5962ef1'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.478559 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ccea5f749a946fb8ef2ec2909f84834', 'request_id': 'req-24482674-bac6-43ad-91b0-a76aa5962ef1'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.485637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ccea5f749a946fb8ef2ec2909f84834', 'request_id': 'req-24482674-bac6-43ad-91b0-a76aa5962ef1'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.497638 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] The token cache is being invalidate because project 6ccea5f749a946fb8ef2ec2909f84834 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:31:57.497945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.498607 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.500081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1444/2862] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:31:57 2022] DELETE /identity/v3/projects/6ccea5f749a946fb8ef2ec2909f84834 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:31:57.504034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934eafd-f7f8-4f8a-82d5-9bdf02c2e7ad service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:57.513422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934eafd-f7f8-4f8a-82d5-9bdf02c2e7ad service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:57.567756 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:57.568373 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:57.568653 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:57.568924 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] PATH_INFO: `/v3/users/7a69bb4798d64edab13fa6d2adc11299` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:57.570053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a69bb4798d64edab13fa6d2adc11299)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:57.571963 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:57.617118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a69bb4798d64edab13fa6d2adc11299'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.624379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a69bb4798d64edab13fa6d2adc11299'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.624995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a69bb4798d64edab13fa6d2adc11299'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.635765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:57.637069 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a69bb4798d64edab13fa6d2adc11299'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.645551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-0a34697b-c227-4395-8e6d-d45292849430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a69bb4798d64edab13fa6d2adc11299'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:57.651384 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1419/2863] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:31:57 2022] DELETE /identity/v3/users/7a69bb4798d64edab13fa6d2adc11299 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:57.909389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:57.910875 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:57.926427 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24482674-bac6-43ad-91b0-a76aa5962ef1 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:57.939232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:57.987209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:57.987825 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:57.988435 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:57.988713 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] PATH_INFO: `/v3/projects/924d0722ad8a4691b153e2dbf68a710a` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:57.989845 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924d0722ad8a4691b153e2dbf68a710a)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:57.991744 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.038225 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924d0722ad8a4691b153e2dbf68a710a', 'request_id': 'req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:58.043834 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924d0722ad8a4691b153e2dbf68a710a', 'request_id': 'req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:58.049986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924d0722ad8a4691b153e2dbf68a710a', 'request_id': 'req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:58.062084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] The token cache is being invalidate because project 924d0722ad8a4691b153e2dbf68a710a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:31:58.062369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:58.062982 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-91ad8dd6-d423-4cfd-b040-8d4b77dbcc55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:58.064445 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1445/2864] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:31:57 2022] DELETE /identity/v3/projects/924d0722ad8a4691b153e2dbf68a710a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:58.069209 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd978b-8c40-46d2-8246-33c1648b39bd None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.069674 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd978b-8c40-46d2-8246-33c1648b39bd None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.069778 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd978b-8c40-46d2-8246-33c1648b39bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.089205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8ebd978b-8c40-46d2-8246-33c1648b39bd None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:58.090410 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8ebd978b-8c40-46d2-8246-33c1648b39bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:58.148718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1420/2865] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:58.156335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d430e-622e-4d4e-84e2-fce79e3d4f73 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.156619 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d430e-622e-4d4e-84e2-fce79e3d4f73 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.156895 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0d430e-622e-4d4e-84e2-fce79e3d4f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.176228 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fe0d430e-622e-4d4e-84e2-fce79e3d4f73 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:58.177465 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fe0d430e-622e-4d4e-84e2-fce79e3d4f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:58.185257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1446/2866] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:58.190054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd978b-8c40-46d2-8246-33c1648b39bd None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.199555 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd978b-8c40-46d2-8246-33c1648b39bd None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.204919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.205509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.205822 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.206095 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] PATH_INFO: `/v3/domains` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.206736 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88442) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:31:58.207038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.208620 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.219871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1421/2867] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:31:58.223851 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d430e-622e-4d4e-84e2-fce79e3d4f73 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.232794 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0d430e-622e-4d4e-84e2-fce79e3d4f73 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.237721 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4cdfb-104c-4bb8-9b43-d3ed686ef055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.238308 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4cdfb-104c-4bb8-9b43-d3ed686ef055 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.238583 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4cdfb-104c-4bb8-9b43-d3ed686ef055 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.238852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4cdfb-104c-4bb8-9b43-d3ed686ef055 admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.239481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4cdfb-104c-4bb8-9b43-d3ed686ef055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.241364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4cdfb-104c-4bb8-9b43-d3ed686ef055 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.268733 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1447/2868] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:58 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:31:58.272581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.282957 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1e992-86aa-43fb-b31a-4d657fdddd03 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.337818 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563d10b-9f70-4b36-a4da-8afb8885ef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.338415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563d10b-9f70-4b36-a4da-8afb8885ef51 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.338693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563d10b-9f70-4b36-a4da-8afb8885ef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.338964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563d10b-9f70-4b36-a4da-8afb8885ef51 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.339590 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563d10b-9f70-4b36-a4da-8afb8885ef51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.341492 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563d10b-9f70-4b36-a4da-8afb8885ef51 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.362941 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1422/2869] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:31:58 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:58.367239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4cdfb-104c-4bb8-9b43-d3ed686ef055 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.377864 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4cdfb-104c-4bb8-9b43-d3ed686ef055 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.382389 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cf615-a1a7-41a8-af0a-52107e341a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.382975 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cf615-a1a7-41a8-af0a-52107e341a67 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.383248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cf615-a1a7-41a8-af0a-52107e341a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.383517 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cf615-a1a7-41a8-af0a-52107e341a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.384150 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cf615-a1a7-41a8-af0a-52107e341a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.385751 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cf615-a1a7-41a8-af0a-52107e341a67 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.396172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1448/2870] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:58.399826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563d10b-9f70-4b36-a4da-8afb8885ef51 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.409397 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563d10b-9f70-4b36-a4da-8afb8885ef51 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.413929 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea55ce-adf7-4dea-bd2c-7a74877f8f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.414509 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea55ce-adf7-4dea-bd2c-7a74877f8f30 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.414787 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea55ce-adf7-4dea-bd2c-7a74877f8f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.415054 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea55ce-adf7-4dea-bd2c-7a74877f8f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.415678 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea55ce-adf7-4dea-bd2c-7a74877f8f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.417249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea55ce-adf7-4dea-bd2c-7a74877f8f30 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.427559 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1423/2871] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:58.431425 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cf615-a1a7-41a8-af0a-52107e341a67 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.441808 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cf615-a1a7-41a8-af0a-52107e341a67 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.446359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023bc6e-3db4-46cd-8965-39c9eccd7798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.447000 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023bc6e-3db4-46cd-8965-39c9eccd7798 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.447276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023bc6e-3db4-46cd-8965-39c9eccd7798 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.447546 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023bc6e-3db4-46cd-8965-39c9eccd7798 admin admin] PATH_INFO: `/v3/projects/ffd23595fdec4708a4606a9a5b718741/users/ee7f8921a947410b84a4b69a5281495c/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.462421 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023bc6e-3db4-46cd-8965-39c9eccd7798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd23595fdec4708a4606a9a5b718741, user_id=ee7f8921a947410b84a4b69a5281495c, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.465075 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023bc6e-3db4-46cd-8965-39c9eccd7798 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.478007 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1449/2872] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:58 2022] PUT /identity/v3/projects/ffd23595fdec4708a4606a9a5b718741/users/ee7f8921a947410b84a4b69a5281495c/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:58.489155 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d57f45-fd25-40b3-959e-dd984ac5cd34 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.489447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d57f45-fd25-40b3-959e-dd984ac5cd34 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.489765 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d57f45-fd25-40b3-959e-dd984ac5cd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.520411 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-89d57f45-fd25-40b3-959e-dd984ac5cd34 None None] MFA Rules not processed for user `ee7f8921a947410b84a4b69a5281495c`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:58.521543 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-89d57f45-fd25-40b3-959e-dd984ac5cd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:58.561219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:58.584744 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1424/2873] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:58.589214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023bc6e-3db4-46cd-8965-39c9eccd7798 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.634520 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:58.653481 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023bc6e-3db4-46cd-8965-39c9eccd7798 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.658203 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51d7e0-f826-44d2-bde3-e3db6e33f3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.658803 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51d7e0-f826-44d2-bde3-e3db6e33f3ac admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.659084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51d7e0-f826-44d2-bde3-e3db6e33f3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.659358 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51d7e0-f826-44d2-bde3-e3db6e33f3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.659991 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51d7e0-f826-44d2-bde3-e3db6e33f3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.661934 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51d7e0-f826-44d2-bde3-e3db6e33f3ac admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.699458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1450/2874] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:58 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:31:58.702940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d57f45-fd25-40b3-959e-dd984ac5cd34 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.714217 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d57f45-fd25-40b3-959e-dd984ac5cd34 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.769741 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8ce98-4949-468f-872d-fd7c37fbdc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.770355 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8ce98-4949-468f-872d-fd7c37fbdc69 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.770685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8ce98-4949-468f-872d-fd7c37fbdc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.770914 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac8ce98-4949-468f-872d-fd7c37fbdc69 admin admin] PATH_INFO: `/v3/users` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.771581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8ce98-4949-468f-872d-fd7c37fbdc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.773501 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac8ce98-4949-468f-872d-fd7c37fbdc69 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.793894 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1425/2875] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:31:58 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:58.798423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51d7e0-f826-44d2-bde3-e3db6e33f3ac admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.808964 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51d7e0-f826-44d2-bde3-e3db6e33f3ac admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.814183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13632b2-1610-4ce8-bd53-5d6b7c49d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.814776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13632b2-1610-4ce8-bd53-5d6b7c49d916 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.815055 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13632b2-1610-4ce8-bd53-5d6b7c49d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.815328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13632b2-1610-4ce8-bd53-5d6b7c49d916 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.815962 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13632b2-1610-4ce8-bd53-5d6b7c49d916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.817542 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13632b2-1610-4ce8-bd53-5d6b7c49d916 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.828179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1451/2876] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:31:58.832035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8ce98-4949-468f-872d-fd7c37fbdc69 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.841654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac8ce98-4949-468f-872d-fd7c37fbdc69 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.846288 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cabeff-c992-4d61-afcb-c86c1871d0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.846960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabeff-c992-4d61-afcb-c86c1871d0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.847230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabeff-c992-4d61-afcb-c86c1871d0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.847506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabeff-c992-4d61-afcb-c86c1871d0c2 admin admin] PATH_INFO: `/v3/projects/ae3ce3daa3104acc8b7e2ceef5f0e759/users/cb3ebddddeb34ac0a7cb510037feedc7/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.863239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cabeff-c992-4d61-afcb-c86c1871d0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3ce3daa3104acc8b7e2ceef5f0e759, user_id=cb3ebddddeb34ac0a7cb510037feedc7, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.865930 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cabeff-c992-4d61-afcb-c86c1871d0c2 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.879493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1426/2877] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:58 2022] PUT /identity/v3/projects/ae3ce3daa3104acc8b7e2ceef5f0e759/users/cb3ebddddeb34ac0a7cb510037feedc7/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:58.883493 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13632b2-1610-4ce8-bd53-5d6b7c49d916 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.893315 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13632b2-1610-4ce8-bd53-5d6b7c49d916 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.948535 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c031e4-24e4-435b-8284-b72401d08edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.949131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c031e4-24e4-435b-8284-b72401d08edb admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.949410 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c031e4-24e4-435b-8284-b72401d08edb admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.949711 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c031e4-24e4-435b-8284-b72401d08edb admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.950346 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c031e4-24e4-435b-8284-b72401d08edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.951921 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c031e4-24e4-435b-8284-b72401d08edb admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.961418 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1452/2878] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:58.965488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cabeff-c992-4d61-afcb-c86c1871d0c2 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:58.976080 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cabeff-c992-4d61-afcb-c86c1871d0c2 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:58.980646 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:58.981368 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:58.981561 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:58.981863 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] PATH_INFO: `/v3/projects/ae3ce3daa3104acc8b7e2ceef5f0e759/users/cb3ebddddeb34ac0a7cb510037feedc7/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:58.983749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3ce3daa3104acc8b7e2ceef5f0e759, user_id=cb3ebddddeb34ac0a7cb510037feedc7, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:58.987306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:58.997233 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1427/2879] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:58 2022] PUT /identity/v3/projects/ae3ce3daa3104acc8b7e2ceef5f0e759/users/cb3ebddddeb34ac0a7cb510037feedc7/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:59.007209 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e5651-3b0c-420d-a224-1a613e42a0d9 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:59.007675 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e5651-3b0c-420d-a224-1a613e42a0d9 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:59.007755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e5651-3b0c-420d-a224-1a613e42a0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:59.039126 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-a19e5651-3b0c-420d-a224-1a613e42a0d9 None None] MFA Rules not processed for user `cb3ebddddeb34ac0a7cb510037feedc7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:59.040276 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a19e5651-3b0c-420d-a224-1a613e42a0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:59.108672 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1453/2880] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:31:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:59.122022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:59.123755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:59.179971 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbfd70-3604-4b67-ac47-7307b4e69a8d admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:59.183615 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d764d9d-fa80-4daf-aeee-b1f279d76295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:59.184205 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d764d9d-fa80-4daf-aeee-b1f279d76295 service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:59.184485 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d764d9d-fa80-4daf-aeee-b1f279d76295 service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:59.184755 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d764d9d-fa80-4daf-aeee-b1f279d76295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:59.193558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d764d9d-fa80-4daf-aeee-b1f279d76295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:59.195047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d764d9d-fa80-4daf-aeee-b1f279d76295 service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:59.204180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1428/2881] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:59.506365 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e5651-3b0c-420d-a224-1a613e42a0d9 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:59.515388 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e5651-3b0c-420d-a224-1a613e42a0d9 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:59.520432 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3ebddddeb34ac0a7cb510037feedc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae3ce3daa3104acc8b7e2ceef5f0e759', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:59.521030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:59.521423 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:59.521793 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:59.522448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:31:59.522770 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:59.524341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:59.536207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1454/2882] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:31:59.540207 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d764d9d-fa80-4daf-aeee-b1f279d76295 service cinder] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:59.549140 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d764d9d-fa80-4daf-aeee-b1f279d76295 service cinder] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:59.553724 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3acb3-9311-4f8b-878a-f5a24e7e2700 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3ebddddeb34ac0a7cb510037feedc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae3ce3daa3104acc8b7e2ceef5f0e759', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:59.554307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3acb3-9311-4f8b-878a-f5a24e7e2700 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:59.554587 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3acb3-9311-4f8b-878a-f5a24e7e2700 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:59.554867 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3acb3-9311-4f8b-878a-f5a24e7e2700 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:59.555497 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3acb3-9311-4f8b-878a-f5a24e7e2700 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:59.557379 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3acb3-9311-4f8b-878a-f5a24e7e2700 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:59.583666 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1429/2883] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:59 2022] POST /identity/v3/projects => generated 354 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:59.588011 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:59.599043 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491903-369e-48af-b67c-fa6e300eed6c tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:59.619485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:59.658305 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3ebddddeb34ac0a7cb510037feedc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae3ce3daa3104acc8b7e2ceef5f0e759', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:59.658910 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:59.659192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:59.659467 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:59.660103 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:31:59.660439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:59.662068 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:59.673809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1455/2884] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:31:59.713732 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3acb3-9311-4f8b-878a-f5a24e7e2700 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:59.724928 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3acb3-9311-4f8b-878a-f5a24e7e2700 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:59.729464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3ebddddeb34ac0a7cb510037feedc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae3ce3daa3104acc8b7e2ceef5f0e759', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:59.730127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:59.730413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:59.730684 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] PATH_INFO: `/v3/projects/7a16a289f7184d44880b3d803aa78720` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:59.731802 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7a16a289f7184d44880b3d803aa78720)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:59.733815 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:59.745093 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:59.783271 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a16a289f7184d44880b3d803aa78720', 'request_id': 'req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:59.788844 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a16a289f7184d44880b3d803aa78720', 'request_id': 'req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:59.795471 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a16a289f7184d44880b3d803aa78720', 'request_id': 'req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:59.807874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] The token cache is being invalidate because project 7a16a289f7184d44880b3d803aa78720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:31:59.808194 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:59.808846 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-d0e15d09-194f-4cd6-aa81-b54142ae6fa8 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:59.810365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1430/2885] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:31:59 2022] DELETE /identity/v3/projects/7a16a289f7184d44880b3d803aa78720 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:59.996335 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:59.998105 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:00.069837 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb687b-eac6-4fbc-886a-8992f1a75134 tempest-VolumeQuotaClassesTest-92916962 tempest-VolumeQuotaClassesTest-92916962-project-admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:00.126888 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:00.127553 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:00.127832 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:00.128120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] PATH_INFO: `/v3/users/ee7f8921a947410b84a4b69a5281495c` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:00.129374 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee7f8921a947410b84a4b69a5281495c)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:00.131364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:00.176543 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee7f8921a947410b84a4b69a5281495c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.183703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee7f8921a947410b84a4b69a5281495c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.184548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee7f8921a947410b84a4b69a5281495c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.195568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:00.198214 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee7f8921a947410b84a4b69a5281495c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.206366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee7f8921a947410b84a4b69a5281495c'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.212419 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1456/2886] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:31:59 2022] DELETE /identity/v3/users/ee7f8921a947410b84a4b69a5281495c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:00.216960 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27b60b-ee35-494a-b636-14ceee339a06 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:00.217453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27b60b-ee35-494a-b636-14ceee339a06 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:00.217546 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27b60b-ee35-494a-b636-14ceee339a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:00.236786 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-ec27b60b-ee35-494a-b636-14ceee339a06 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:00.238012 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ec27b60b-ee35-494a-b636-14ceee339a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:00.293668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1431/2887] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:00.303649 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:00.305037 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:00.361660 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafb7c2-fb2a-434e-a4bc-a7688c66ce1e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:00.365314 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d42722-0955-46b8-add0-1d378d0f9533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:00.365940 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d42722-0955-46b8-add0-1d378d0f9533 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:00.366207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d42722-0955-46b8-add0-1d378d0f9533 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:00.366475 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d42722-0955-46b8-add0-1d378d0f9533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:00.376338 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d42722-0955-46b8-add0-1d378d0f9533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:00.377840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d42722-0955-46b8-add0-1d378d0f9533 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:00.387376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1457/2888] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:00.648415 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27b60b-ee35-494a-b636-14ceee339a06 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:00.649873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-ec27b60b-ee35-494a-b636-14ceee339a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:00.662413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27b60b-ee35-494a-b636-14ceee339a06 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:00.667025 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:00.667662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:00.667945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:00.668222 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] PATH_INFO: `/v3/projects/ffd23595fdec4708a4606a9a5b718741` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:00.669339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd23595fdec4708a4606a9a5b718741)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:00.671307 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:00.717915 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd23595fdec4708a4606a9a5b718741', 'request_id': 'req-63585545-77fa-411c-9ec0-f8e99702a60b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.723990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd23595fdec4708a4606a9a5b718741', 'request_id': 'req-63585545-77fa-411c-9ec0-f8e99702a60b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.730111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd23595fdec4708a4606a9a5b718741', 'request_id': 'req-63585545-77fa-411c-9ec0-f8e99702a60b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.742837 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] The token cache is being invalidate because project ffd23595fdec4708a4606a9a5b718741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:32:00.743124 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.743752 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.745230 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1432/2889] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:32:00 2022] DELETE /identity/v3/projects/ffd23595fdec4708a4606a9a5b718741 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:00.749182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d42722-0955-46b8-add0-1d378d0f9533 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:00.759175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d42722-0955-46b8-add0-1d378d0f9533 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:00.815074 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:00.815694 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:00.815972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:00.816243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] PATH_INFO: `/v3/users/cb3ebddddeb34ac0a7cb510037feedc7` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:00.817366 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3ebddddeb34ac0a7cb510037feedc7)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:00.819325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:00.864796 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3ebddddeb34ac0a7cb510037feedc7'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.871802 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3ebddddeb34ac0a7cb510037feedc7'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.872505 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3ebddddeb34ac0a7cb510037feedc7'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.884819 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3ebddddeb34ac0a7cb510037feedc7'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.892200 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-ceab6974-1f77-4d4a-b832-2e6e2e8422db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3ebddddeb34ac0a7cb510037feedc7'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:00.898097 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1458/2890] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:32:00 2022] DELETE /identity/v3/users/cb3ebddddeb34ac0a7cb510037feedc7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:01.164182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.165632 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:01.179764 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63585545-77fa-411c-9ec0-f8e99702a60b admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:01.199129 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:01.238956 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:01.239591 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.239841 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.240111 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] PATH_INFO: `/v3/projects/ae3ce3daa3104acc8b7e2ceef5f0e759` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.241259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3ce3daa3104acc8b7e2ceef5f0e759)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:01.243150 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:01.287716 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3ce3daa3104acc8b7e2ceef5f0e759', 'request_id': 'req-a71359af-d113-4b7b-96c8-ee8aa5c8e514'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:01.292991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3ce3daa3104acc8b7e2ceef5f0e759', 'request_id': 'req-a71359af-d113-4b7b-96c8-ee8aa5c8e514'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:01.299588 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3ce3daa3104acc8b7e2ceef5f0e759', 'request_id': 'req-a71359af-d113-4b7b-96c8-ee8aa5c8e514'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:01.311895 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] The token cache is being invalidate because project ae3ce3daa3104acc8b7e2ceef5f0e759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:32:01.312366 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:01.312989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-a71359af-d113-4b7b-96c8-ee8aa5c8e514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:01.314494 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1433/2891] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:32:01 2022] DELETE /identity/v3/projects/ae3ce3daa3104acc8b7e2ceef5f0e759 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:32:01.319313 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ebd7-3812-44e5-b921-a2e3377fd331 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.319788 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ebd7-3812-44e5-b921-a2e3377fd331 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.319887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ebd7-3812-44e5-b921-a2e3377fd331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.333355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:01.340223 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-d8d4ebd7-3812-44e5-b921-a2e3377fd331 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:01.341997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-d8d4ebd7-3812-44e5-b921-a2e3377fd331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:01.402548 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1459/2892] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:01.410719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599921a9-4e8b-4dda-8e9d-ddcc7568c55e None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.411004 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599921a9-4e8b-4dda-8e9d-ddcc7568c55e None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.411278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599921a9-4e8b-4dda-8e9d-ddcc7568c55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.431316 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-599921a9-4e8b-4dda-8e9d-ddcc7568c55e None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:01.432498 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-599921a9-4e8b-4dda-8e9d-ddcc7568c55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:01.440462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1434/2893] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:01.444690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4ebd7-3812-44e5-b921-a2e3377fd331 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.454416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4ebd7-3812-44e5-b921-a2e3377fd331 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:01.459013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:01.459602 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.459880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.460152 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.460790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:32:01.461092 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:01.462785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:01.474722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1460/2894] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:32:01.478849 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599921a9-4e8b-4dda-8e9d-ddcc7568c55e None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.488781 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599921a9-4e8b-4dda-8e9d-ddcc7568c55e None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:01.493449 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50c70a-77b4-44b4-b437-59d46e22e020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:01.494011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50c70a-77b4-44b4-b437-59d46e22e020 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.494291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50c70a-77b4-44b4-b437-59d46e22e020 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.494558 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50c70a-77b4-44b4-b437-59d46e22e020 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.495278 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50c70a-77b4-44b4-b437-59d46e22e020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:01.497313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50c70a-77b4-44b4-b437-59d46e22e020 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:01.526204 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1435/2895] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:32:01 2022] POST /identity/v3/projects => generated 409 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:01.530331 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.541018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bd4dd-6e0d-4617-a171-80dd702a09d3 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:01.595645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f93a0-d045-485f-950d-30f99e80315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:01.596243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f93a0-d045-485f-950d-30f99e80315e admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.596527 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f93a0-d045-485f-950d-30f99e80315e admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.596790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6f93a0-d045-485f-950d-30f99e80315e admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.597446 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f93a0-d045-485f-950d-30f99e80315e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:01.599364 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6f93a0-d045-485f-950d-30f99e80315e admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:01.620147 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1461/2896] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:32:01 2022] POST /identity/v3/users => generated 305 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:01.624441 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50c70a-77b4-44b4-b437-59d46e22e020 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.634518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50c70a-77b4-44b4-b437-59d46e22e020 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:01.638989 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2513df-bb2b-46de-af49-0207eddedc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:01.639571 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2513df-bb2b-46de-af49-0207eddedc6c admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.639850 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2513df-bb2b-46de-af49-0207eddedc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.640122 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2513df-bb2b-46de-af49-0207eddedc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.641358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2513df-bb2b-46de-af49-0207eddedc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:01.642979 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2513df-bb2b-46de-af49-0207eddedc6c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:01.652760 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1436/2897] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:01.657281 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f93a0-d045-485f-950d-30f99e80315e admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.667155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6f93a0-d045-485f-950d-30f99e80315e admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:01.671763 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17239ee0-e889-478b-8958-d63e7b795e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:01.672350 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17239ee0-e889-478b-8958-d63e7b795e95 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.672632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17239ee0-e889-478b-8958-d63e7b795e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.672908 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17239ee0-e889-478b-8958-d63e7b795e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.673549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17239ee0-e889-478b-8958-d63e7b795e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:01.675160 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17239ee0-e889-478b-8958-d63e7b795e95 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:01.685504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1462/2898] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:01.689493 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2513df-bb2b-46de-af49-0207eddedc6c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.699291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2513df-bb2b-46de-af49-0207eddedc6c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:01.704338 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf828da1-8426-455e-b7f2-89b61c3c2d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:01.704969 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf828da1-8426-455e-b7f2-89b61c3c2d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.705247 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf828da1-8426-455e-b7f2-89b61c3c2d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.705518 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf828da1-8426-455e-b7f2-89b61c3c2d96 admin admin] PATH_INFO: `/v3/projects/667cb7569a564217a6e28866bc25617a/users/587becbf1e474d75a1b3653dd02f49ea/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.722931 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf828da1-8426-455e-b7f2-89b61c3c2d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=667cb7569a564217a6e28866bc25617a, user_id=587becbf1e474d75a1b3653dd02f49ea, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:01.725645 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf828da1-8426-455e-b7f2-89b61c3c2d96 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:01.738260 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1437/2899] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:32:01 2022] PUT /identity/v3/projects/667cb7569a564217a6e28866bc25617a/users/587becbf1e474d75a1b3653dd02f49ea/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:01.749496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6fed-fde6-488f-9f44-a3ab182ed2a7 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.749824 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6fed-fde6-488f-9f44-a3ab182ed2a7 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.750098 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c6fed-fde6-488f-9f44-a3ab182ed2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.778795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-ba3c6fed-fde6-488f-9f44-a3ab182ed2a7 None None] MFA Rules not processed for user `587becbf1e474d75a1b3653dd02f49ea`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:01.780010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-ba3c6fed-fde6-488f-9f44-a3ab182ed2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:01.842669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1463/2900] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:01 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:01.847640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf828da1-8426-455e-b7f2-89b61c3c2d96 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.896202 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:01.911199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf828da1-8426-455e-b7f2-89b61c3c2d96 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:01.915813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bdd52-4607-4117-8221-b6ca805d77b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:01.916417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bdd52-4607-4117-8221-b6ca805d77b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:01.916699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bdd52-4607-4117-8221-b6ca805d77b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:01.916973 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bdd52-4607-4117-8221-b6ca805d77b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:01.917640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bdd52-4607-4117-8221-b6ca805d77b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:01.919580 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bdd52-4607-4117-8221-b6ca805d77b7 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:01.946519 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1438/2901] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:32:01 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:01.950567 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c6fed-fde6-488f-9f44-a3ab182ed2a7 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:01.956556 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:01.963566 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c6fed-fde6-488f-9f44-a3ab182ed2a7 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:02.020560 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ae94-e3ba-49e0-9305-76246aea76bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:02.021179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ae94-e3ba-49e0-9305-76246aea76bd admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:02.021458 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ae94-e3ba-49e0-9305-76246aea76bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:02.021757 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ae94-e3ba-49e0-9305-76246aea76bd admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:02.022385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372ae94-e3ba-49e0-9305-76246aea76bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:02.024304 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372ae94-e3ba-49e0-9305-76246aea76bd admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:02.044443 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1464/2902] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:32:01 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:02.048395 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bdd52-4607-4117-8221-b6ca805d77b7 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:02.058514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bdd52-4607-4117-8221-b6ca805d77b7 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:02.063589 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a2b00-5d8c-4073-9bff-6011e15db724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:02.064212 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2b00-5d8c-4073-9bff-6011e15db724 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:02.064489 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2b00-5d8c-4073-9bff-6011e15db724 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:02.064763 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a2b00-5d8c-4073-9bff-6011e15db724 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:02.065396 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a2b00-5d8c-4073-9bff-6011e15db724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:02.067034 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a2b00-5d8c-4073-9bff-6011e15db724 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:02.077311 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1439/2903] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:02.081812 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ae94-e3ba-49e0-9305-76246aea76bd admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:02.092023 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ae94-e3ba-49e0-9305-76246aea76bd admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:02.096748 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992b65a-e947-4cca-82cd-39c659bc6ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:02.097385 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992b65a-e947-4cca-82cd-39c659bc6ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:02.097704 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992b65a-e947-4cca-82cd-39c659bc6ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:02.097983 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992b65a-e947-4cca-82cd-39c659bc6ac0 admin admin] PATH_INFO: `/v3/projects/0779831ed95348a989b934159ad764a9/users/eb2ca418be71490fab8b0ff8f8ff9541/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:02.112753 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992b65a-e947-4cca-82cd-39c659bc6ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0779831ed95348a989b934159ad764a9, user_id=eb2ca418be71490fab8b0ff8f8ff9541, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:02.115595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992b65a-e947-4cca-82cd-39c659bc6ac0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:02.128876 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1465/2904] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:32:02 2022] PUT /identity/v3/projects/0779831ed95348a989b934159ad764a9/users/eb2ca418be71490fab8b0ff8f8ff9541/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:02.132726 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a2b00-5d8c-4073-9bff-6011e15db724 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:02.143118 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a2b00-5d8c-4073-9bff-6011e15db724 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:02.197431 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efffb4-3c77-4c97-9925-10f877bd4be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:02.198067 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efffb4-3c77-4c97-9925-10f877bd4be5 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:02.198356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efffb4-3c77-4c97-9925-10f877bd4be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:02.198629 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efffb4-3c77-4c97-9925-10f877bd4be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:02.199264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efffb4-3c77-4c97-9925-10f877bd4be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:02.200866 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efffb4-3c77-4c97-9925-10f877bd4be5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:02.211022 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1440/2905] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:02.215138 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992b65a-e947-4cca-82cd-39c659bc6ac0 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:02.225591 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992b65a-e947-4cca-82cd-39c659bc6ac0 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:02.230164 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:02.230809 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:02.231084 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:02.231376 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] PATH_INFO: `/v3/projects/0779831ed95348a989b934159ad764a9/users/eb2ca418be71490fab8b0ff8f8ff9541/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:02.233285 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0779831ed95348a989b934159ad764a9, user_id=eb2ca418be71490fab8b0ff8f8ff9541, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:02.236261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:02.246172 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1466/2906] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:32:02 2022] PUT /identity/v3/projects/0779831ed95348a989b934159ad764a9/users/eb2ca418be71490fab8b0ff8f8ff9541/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:02.256402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a54a-dda7-40ee-b213-723d4f838431 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:02.256858 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a54a-dda7-40ee-b213-723d4f838431 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:02.256945 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858a54a-dda7-40ee-b213-723d4f838431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:02.284206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-5858a54a-dda7-40ee-b213-723d4f838431 None None] MFA Rules not processed for user `eb2ca418be71490fab8b0ff8f8ff9541`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:02.285324 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-5858a54a-dda7-40ee-b213-723d4f838431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:02.352143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1441/2907] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:02.363690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:02.365436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:02.423587 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dcab12-fe6b-4c4d-b3a0-94eb8bfd6922 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:02.429760 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607839f4-c9ed-4fa9-9fd6-dbc6c68e8856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:02.430373 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607839f4-c9ed-4fa9-9fd6-dbc6c68e8856 service cinder] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:02.430653 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607839f4-c9ed-4fa9-9fd6-dbc6c68e8856 service cinder] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:02.430927 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607839f4-c9ed-4fa9-9fd6-dbc6c68e8856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:02.440620 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607839f4-c9ed-4fa9-9fd6-dbc6c68e8856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:02.442086 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607839f4-c9ed-4fa9-9fd6-dbc6c68e8856 service cinder] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:02.451790 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1467/2908] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:02 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:02.600394 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858a54a-dda7-40ee-b213-723d4f838431 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:02.610896 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858a54a-dda7-40ee-b213-723d4f838431 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:02.614446 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5712e3-aecf-446f-8c98-1dcf02fe6baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:02.615068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5712e3-aecf-446f-8c98-1dcf02fe6baa service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:02.615412 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5712e3-aecf-446f-8c98-1dcf02fe6baa service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:02.615705 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5712e3-aecf-446f-8c98-1dcf02fe6baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:02.657289 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:02.673175 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5712e3-aecf-446f-8c98-1dcf02fe6baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:02.674638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5712e3-aecf-446f-8c98-1dcf02fe6baa service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:02.683514 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1442/2909] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:02 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:11.007496 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607839f4-c9ed-4fa9-9fd6-dbc6c68e8856 service cinder] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:11.066486 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:11.070779 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607839f4-c9ed-4fa9-9fd6-dbc6c68e8856 service cinder] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:11.076549 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:11.077176 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:11.077470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:11.077764 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] PATH_INFO: `/v3/users/587becbf1e474d75a1b3653dd02f49ea` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:11.078897 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=587becbf1e474d75a1b3653dd02f49ea)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:11.080861 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:11.127159 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '587becbf1e474d75a1b3653dd02f49ea'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.133887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '587becbf1e474d75a1b3653dd02f49ea'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.134510 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '587becbf1e474d75a1b3653dd02f49ea'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.146477 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '587becbf1e474d75a1b3653dd02f49ea'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.153236 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '587becbf1e474d75a1b3653dd02f49ea'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.159589 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1468/2910] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:32:11 2022] DELETE /identity/v3/users/587becbf1e474d75a1b3653dd02f49ea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:11.164038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6ec3-c619-4ec9-b799-ad2a526145c2 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:11.164461 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6ec3-c619-4ec9-b799-ad2a526145c2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:11.164640 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde6ec3-c619-4ec9-b799-ad2a526145c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:11.184156 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-3dde6ec3-c619-4ec9-b799-ad2a526145c2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:11.185358 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3dde6ec3-c619-4ec9-b799-ad2a526145c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:11.242616 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1443/2911] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:11.252182 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:11.253570 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:11.309852 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb922ba-13ad-40bc-ac6b-12d595c67435 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:11.313490 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a936f2a-49d6-424d-9337-1ce03a6cfca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:11.314118 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a936f2a-49d6-424d-9337-1ce03a6cfca6 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:11.314397 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a936f2a-49d6-424d-9337-1ce03a6cfca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:11.314669 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a936f2a-49d6-424d-9337-1ce03a6cfca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:11.323552 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a936f2a-49d6-424d-9337-1ce03a6cfca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:11.324971 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a936f2a-49d6-424d-9337-1ce03a6cfca6 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:11.334786 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1469/2912] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:11.597019 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde6ec3-c619-4ec9-b799-ad2a526145c2 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:11.598476 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-3dde6ec3-c619-4ec9-b799-ad2a526145c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:11.610393 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde6ec3-c619-4ec9-b799-ad2a526145c2 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:11.615101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:11.615748 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:11.617056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:11.617056 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] PATH_INFO: `/v3/projects/667cb7569a564217a6e28866bc25617a` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:11.617448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=667cb7569a564217a6e28866bc25617a)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:11.619464 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:11.669113 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '667cb7569a564217a6e28866bc25617a', 'request_id': 'req-83ee0d4b-1885-4850-b695-841df4109686'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.675794 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '667cb7569a564217a6e28866bc25617a', 'request_id': 'req-83ee0d4b-1885-4850-b695-841df4109686'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.681421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '667cb7569a564217a6e28866bc25617a', 'request_id': 'req-83ee0d4b-1885-4850-b695-841df4109686'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.694092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] The token cache is being invalidate because project 667cb7569a564217a6e28866bc25617a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:32:11.694376 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.694983 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.696447 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1444/2913] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:32:11 2022] DELETE /identity/v3/projects/667cb7569a564217a6e28866bc25617a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:11.700474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a936f2a-49d6-424d-9337-1ce03a6cfca6 service neutron] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:11.710729 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a936f2a-49d6-424d-9337-1ce03a6cfca6 service neutron] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:11.769966 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:11.770597 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:11.770871 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:11.771140 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] PATH_INFO: `/v3/users/eb2ca418be71490fab8b0ff8f8ff9541` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:11.772253 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2ca418be71490fab8b0ff8f8ff9541)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:11.774204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:11.795207 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:11.823067 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2ca418be71490fab8b0ff8f8ff9541'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.832502 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2ca418be71490fab8b0ff8f8ff9541'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.833121 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2ca418be71490fab8b0ff8f8ff9541'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.845336 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2ca418be71490fab8b0ff8f8ff9541'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.852033 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-071e9b29-e175-472e-a4d2-b6ff8095b05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2ca418be71490fab8b0ff8f8ff9541'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:11.857771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1470/2914] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:32:11 2022] DELETE /identity/v3/users/eb2ca418be71490fab8b0ff8f8ff9541 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:12.128123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:12.129598 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:12.142574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee0d4b-1885-4850-b695-841df4109686 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:12.199422 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:12.200077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.200356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.200628 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] PATH_INFO: `/v3/projects/0779831ed95348a989b934159ad764a9` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.201811 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0779831ed95348a989b934159ad764a9)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:12.203656 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:12.211219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:12.249127 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0779831ed95348a989b934159ad764a9', 'request_id': 'req-82814d3a-ad44-4215-bab3-fc4cd8ded247'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:12.255267 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0779831ed95348a989b934159ad764a9', 'request_id': 'req-82814d3a-ad44-4215-bab3-fc4cd8ded247'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:12.261151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0779831ed95348a989b934159ad764a9', 'request_id': 'req-82814d3a-ad44-4215-bab3-fc4cd8ded247'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:12.272792 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] The token cache is being invalidate because project 0779831ed95348a989b934159ad764a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:32:12.273076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:12.273728 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-82814d3a-ad44-4215-bab3-fc4cd8ded247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:12.275180 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1445/2915] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:32:12 2022] DELETE /identity/v3/projects/0779831ed95348a989b934159ad764a9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:12.280057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770e89-e5d7-40b0-a76f-a7e88f7386d5 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.280544 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770e89-e5d7-40b0-a76f-a7e88f7386d5 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.280636 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770e89-e5d7-40b0-a76f-a7e88f7386d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.300059 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-bb770e89-e5d7-40b0-a76f-a7e88f7386d5 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:12.301243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-bb770e89-e5d7-40b0-a76f-a7e88f7386d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:12.359731 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1471/2916] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:12.369090 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a60fd4-b87b-46c7-9dac-b6bb0cbb08f7 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.369380 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a60fd4-b87b-46c7-9dac-b6bb0cbb08f7 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.369697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a60fd4-b87b-46c7-9dac-b6bb0cbb08f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.389064 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-d5a60fd4-b87b-46c7-9dac-b6bb0cbb08f7 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:12.390302 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d5a60fd4-b87b-46c7-9dac-b6bb0cbb08f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:12.398144 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1446/2917] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:12.402073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb770e89-e5d7-40b0-a76f-a7e88f7386d5 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:12.412411 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb770e89-e5d7-40b0-a76f-a7e88f7386d5 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:12.416988 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:12.417592 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.417881 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.418155 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.418785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:32:12.419081 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:12.420652 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:12.431906 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1472/2918] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:32:12.436071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a60fd4-b87b-46c7-9dac-b6bb0cbb08f7 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:12.445855 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a60fd4-b87b-46c7-9dac-b6bb0cbb08f7 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:12.450453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726360c-0393-4517-85a9-a742d41e9b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:12.451038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726360c-0393-4517-85a9-a742d41e9b8c admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.451313 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726360c-0393-4517-85a9-a742d41e9b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.451586 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726360c-0393-4517-85a9-a742d41e9b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.452268 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726360c-0393-4517-85a9-a742d41e9b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:12.454823 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726360c-0393-4517-85a9-a742d41e9b8c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:12.480697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1447/2919] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:32:12 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:12.485018 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:12.494968 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1cd43-4f68-4168-9af1-dd6499a3e137 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:12.550574 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b636a-da8e-4a8c-ab5a-52110e2917d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:12.551175 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b636a-da8e-4a8c-ab5a-52110e2917d7 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.551453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b636a-da8e-4a8c-ab5a-52110e2917d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.551722 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b636a-da8e-4a8c-ab5a-52110e2917d7 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.552354 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b636a-da8e-4a8c-ab5a-52110e2917d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:12.554307 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b636a-da8e-4a8c-ab5a-52110e2917d7 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:12.575944 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1473/2920] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:32:12 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:12.580544 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726360c-0393-4517-85a9-a742d41e9b8c admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:12.590788 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726360c-0393-4517-85a9-a742d41e9b8c admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:12.595418 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f055e7e-b7e1-4d11-bb1a-f0f478d97cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:12.596013 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f055e7e-b7e1-4d11-bb1a-f0f478d97cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.596290 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f055e7e-b7e1-4d11-bb1a-f0f478d97cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.596563 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f055e7e-b7e1-4d11-bb1a-f0f478d97cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.597197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f055e7e-b7e1-4d11-bb1a-f0f478d97cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:12.598826 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f055e7e-b7e1-4d11-bb1a-f0f478d97cc0 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:12.608490 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1448/2921] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:12.612471 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b636a-da8e-4a8c-ab5a-52110e2917d7 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:12.622179 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b636a-da8e-4a8c-ab5a-52110e2917d7 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:12.627539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7387b1-d8de-44f8-ba59-a90226f08977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:12.628122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7387b1-d8de-44f8-ba59-a90226f08977 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.628398 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7387b1-d8de-44f8-ba59-a90226f08977 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.628667 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7387b1-d8de-44f8-ba59-a90226f08977 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.629298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7387b1-d8de-44f8-ba59-a90226f08977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:12.630922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7387b1-d8de-44f8-ba59-a90226f08977 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:12.642830 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1474/2922] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:12.646919 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f055e7e-b7e1-4d11-bb1a-f0f478d97cc0 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:12.656545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f055e7e-b7e1-4d11-bb1a-f0f478d97cc0 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:12.661146 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:12.661881 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.662166 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.662439 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] PATH_INFO: `/v3/projects/2d0ecadbf29241b9b6ac063d0b004198/users/264a5e2f87e8479b99e8309892b46b6e/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.676698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0ecadbf29241b9b6ac063d0b004198, user_id=264a5e2f87e8479b99e8309892b46b6e, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:12.679413 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:12.692604 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1449/2923] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:32:12 2022] PUT /identity/v3/projects/2d0ecadbf29241b9b6ac063d0b004198/users/264a5e2f87e8479b99e8309892b46b6e/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:32:12.702750 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3ce6-072f-434e-8d75-e4e572c875bb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.702974 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3ce6-072f-434e-8d75-e4e572c875bb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.703244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb3ce6-072f-434e-8d75-e4e572c875bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.712090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:12.732436 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-0dcb3ce6-072f-434e-8d75-e4e572c875bb None None] MFA Rules not processed for user `264a5e2f87e8479b99e8309892b46b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:12.733926 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-0dcb3ce6-072f-434e-8d75-e4e572c875bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:12.798791 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1475/2924] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:12.809599 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:12.811291 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:12.866927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe7617-6707-4f67-a983-fd88c8bfb2ce admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:12.870568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27778c8c-8146-4c45-bf4a-80a7c3387f0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:12.871163 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27778c8c-8146-4c45-bf4a-80a7c3387f0c service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:12.871444 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27778c8c-8146-4c45-bf4a-80a7c3387f0c service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:12.871719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27778c8c-8146-4c45-bf4a-80a7c3387f0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:12.880551 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27778c8c-8146-4c45-bf4a-80a7c3387f0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:12.882010 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27778c8c-8146-4c45-bf4a-80a7c3387f0c service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:12.890638 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1450/2925] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:21.274530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3ce6-072f-434e-8d75-e4e572c875bb None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:21.334123 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb3ce6-072f-434e-8d75-e4e572c875bb None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:21.338580 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:21.339192 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:21.339466 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:21.339730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] PATH_INFO: `/v3/users/264a5e2f87e8479b99e8309892b46b6e` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:21.340810 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=264a5e2f87e8479b99e8309892b46b6e)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:21.342742 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:21.387848 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '264a5e2f87e8479b99e8309892b46b6e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.395212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '264a5e2f87e8479b99e8309892b46b6e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.395913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '264a5e2f87e8479b99e8309892b46b6e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.407997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '264a5e2f87e8479b99e8309892b46b6e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.414771 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '264a5e2f87e8479b99e8309892b46b6e'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.420545 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1476/2926] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:32:21 2022] DELETE /identity/v3/users/264a5e2f87e8479b99e8309892b46b6e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:21.424950 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036340f6-fae8-4b6d-8103-670e4a2dd8e4 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:21.425417 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036340f6-fae8-4b6d-8103-670e4a2dd8e4 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:21.425512 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036340f6-fae8-4b6d-8103-670e4a2dd8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:21.444805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-036340f6-fae8-4b6d-8103-670e4a2dd8e4 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:21.446030 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-036340f6-fae8-4b6d-8103-670e4a2dd8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:21.475287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:21.505719 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1451/2927] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:21.515521 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:21.517798 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:21.535050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:21.579755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa743b37-d1ed-4a70-9a2c-8aeac7d37bf0 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:21.583420 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4d362-0861-4f9e-9109-55a67c526009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:21.584020 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d362-0861-4f9e-9109-55a67c526009 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:21.584302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d362-0861-4f9e-9109-55a67c526009 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:21.584575 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d362-0861-4f9e-9109-55a67c526009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:21.593459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4d362-0861-4f9e-9109-55a67c526009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:21.594945 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4d362-0861-4f9e-9109-55a67c526009 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:21.604255 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1477/2928] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:21.862532 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036340f6-fae8-4b6d-8103-670e4a2dd8e4 None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:21.864011 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-036340f6-fae8-4b6d-8103-670e4a2dd8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:21.876513 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036340f6-fae8-4b6d-8103-670e4a2dd8e4 None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:21.882488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:21.883125 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:21.883403 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:21.883677 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] PATH_INFO: `/v3/projects/2d0ecadbf29241b9b6ac063d0b004198` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:21.884809 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d0ecadbf29241b9b6ac063d0b004198)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:21.889644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:21.939718 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d0ecadbf29241b9b6ac063d0b004198', 'request_id': 'req-92414f02-e973-415d-a037-9e38770dba29'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.945503 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d0ecadbf29241b9b6ac063d0b004198', 'request_id': 'req-92414f02-e973-415d-a037-9e38770dba29'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.951771 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d0ecadbf29241b9b6ac063d0b004198', 'request_id': 'req-92414f02-e973-415d-a037-9e38770dba29'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.963790 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] The token cache is being invalidate because project 2d0ecadbf29241b9b6ac063d0b004198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:32:21.964109 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.964751 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-92414f02-e973-415d-a037-9e38770dba29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:21.966264 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1452/2929] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:32:21 2022] DELETE /identity/v3/projects/2d0ecadbf29241b9b6ac063d0b004198 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:21.971860 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f9d34-7bfa-4d94-8b7b-0b534b105377 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:21.972154 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f9d34-7bfa-4d94-8b7b-0b534b105377 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:21.972431 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f9d34-7bfa-4d94-8b7b-0b534b105377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:21.992204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-2e7f9d34-7bfa-4d94-8b7b-0b534b105377 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:21.993407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-2e7f9d34-7bfa-4d94-8b7b-0b534b105377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:22.055627 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1478/2930] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:22.064050 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a117f7a-fe8b-495a-b9fc-46d920ffb1eb None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.064339 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a117f7a-fe8b-495a-b9fc-46d920ffb1eb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.064612 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a117f7a-fe8b-495a-b9fc-46d920ffb1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.083888 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-8a117f7a-fe8b-495a-b9fc-46d920ffb1eb None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:22.085076 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-8a117f7a-fe8b-495a-b9fc-46d920ffb1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:22.092857 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1453/2931] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:22.097447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f9d34-7bfa-4d94-8b7b-0b534b105377 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:22.106898 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f9d34-7bfa-4d94-8b7b-0b534b105377 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:22.112359 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:22.112950 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.113224 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.113494 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.114171 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:32:22.114470 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:22.116050 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:22.127639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1479/2932] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:32:22.131611 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a117f7a-fe8b-495a-b9fc-46d920ffb1eb None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:22.140593 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a117f7a-fe8b-495a-b9fc-46d920ffb1eb None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:22.145199 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd87c6f-7ee8-4cff-aab6-938b5248c375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:22.145814 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87c6f-7ee8-4cff-aab6-938b5248c375 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.146097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87c6f-7ee8-4cff-aab6-938b5248c375 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.146371 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87c6f-7ee8-4cff-aab6-938b5248c375 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.147033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd87c6f-7ee8-4cff-aab6-938b5248c375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:22.148927 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd87c6f-7ee8-4cff-aab6-938b5248c375 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:22.176275 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1454/2933] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:32:22 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:22.180326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:22.190288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b432c8a-23c6-47a2-9c9d-8e6bed0a5dab admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:22.246500 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3b5cb-5c00-4082-8c50-973b1c3a96d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:22.247099 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b5cb-5c00-4082-8c50-973b1c3a96d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.247375 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b5cb-5c00-4082-8c50-973b1c3a96d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.247645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3b5cb-5c00-4082-8c50-973b1c3a96d2 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.248273 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3b5cb-5c00-4082-8c50-973b1c3a96d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:22.250235 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3b5cb-5c00-4082-8c50-973b1c3a96d2 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:22.271248 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1480/2934] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:32:22 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:32:22.275367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd87c6f-7ee8-4cff-aab6-938b5248c375 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:22.286100 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd87c6f-7ee8-4cff-aab6-938b5248c375 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:22.290655 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed05508-d05a-4833-8a75-6f2612702472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:22.291246 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed05508-d05a-4833-8a75-6f2612702472 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.291528 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed05508-d05a-4833-8a75-6f2612702472 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.291803 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed05508-d05a-4833-8a75-6f2612702472 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.292440 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed05508-d05a-4833-8a75-6f2612702472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:22.294108 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed05508-d05a-4833-8a75-6f2612702472 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:22.304402 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1455/2935] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:22.308042 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3b5cb-5c00-4082-8c50-973b1c3a96d2 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:22.317820 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3b5cb-5c00-4082-8c50-973b1c3a96d2 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:22.322428 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bb636-aeaa-4d94-ae86-6d21051b281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:22.323010 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bb636-aeaa-4d94-ae86-6d21051b281c admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.323288 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bb636-aeaa-4d94-ae86-6d21051b281c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.323565 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bb636-aeaa-4d94-ae86-6d21051b281c admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.324194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bb636-aeaa-4d94-ae86-6d21051b281c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:22.325828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bb636-aeaa-4d94-ae86-6d21051b281c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:22.336112 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1481/2936] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:22.340026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed05508-d05a-4833-8a75-6f2612702472 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:22.350269 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed05508-d05a-4833-8a75-6f2612702472 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:22.354813 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:22.355438 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.355714 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.355986 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] PATH_INFO: `/v3/projects/1f44713cf1b946ec980de91554b3988e/users/e3818985977b4705961415c762f2c215/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.371773 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f44713cf1b946ec980de91554b3988e, user_id=e3818985977b4705961415c762f2c215, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:22.374448 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:22.386890 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1456/2937] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:32:22 2022] PUT /identity/v3/projects/1f44713cf1b946ec980de91554b3988e/users/e3818985977b4705961415c762f2c215/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:22.398073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020198d-dc3f-4a91-9a7e-05f9a5d4c7eb None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.398360 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020198d-dc3f-4a91-9a7e-05f9a5d4c7eb None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.398637 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7020198d-dc3f-4a91-9a7e-05f9a5d4c7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.427284 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-7020198d-dc3f-4a91-9a7e-05f9a5d4c7eb None None] MFA Rules not processed for user `e3818985977b4705961415c762f2c215`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:22.428504 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-7020198d-dc3f-4a91-9a7e-05f9a5d4c7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:22.438002 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:22.500320 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1482/2938] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:22.511255 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:22.513101 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:22.570239 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8d0a1-8e01-462a-a5a8-1f7bfd9f9d14 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:22.573789 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89b4ea-bc5f-4c9e-9abc-cf816fb950fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '284740de651e4065845e945556a05ff4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:22.574387 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89b4ea-bc5f-4c9e-9abc-cf816fb950fa service cinder] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:22.574665 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89b4ea-bc5f-4c9e-9abc-cf816fb950fa service cinder] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:22.574940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89b4ea-bc5f-4c9e-9abc-cf816fb950fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:22.583654 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f89b4ea-bc5f-4c9e-9abc-cf816fb950fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:22.585087 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f89b4ea-bc5f-4c9e-9abc-cf816fb950fa service cinder] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:22.588515 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:22.595936 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1457/2939] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:32.497257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=22 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:32.598696 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:32.730785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020198d-dc3f-4a91-9a7e-05f9a5d4c7eb None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:32.794213 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7020198d-dc3f-4a91-9a7e-05f9a5d4c7eb None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:32.798785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:32.799442 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:32.799727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:32.799995 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] PATH_INFO: `/v3/users/e3818985977b4705961415c762f2c215` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:32.801120 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3818985977b4705961415c762f2c215)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:32.803070 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:32.849186 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3818985977b4705961415c762f2c215'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:32.855828 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3818985977b4705961415c762f2c215'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:32.856463 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3818985977b4705961415c762f2c215'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:32.868557 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3818985977b4705961415c762f2c215'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:32.875344 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3818985977b4705961415c762f2c215'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:32.881776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1483/2940] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:32:32 2022] DELETE /identity/v3/users/e3818985977b4705961415c762f2c215 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:32:32.886197 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9f96-5477-4d97-952c-e441e1b29c2b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:32.886482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9f96-5477-4d97-952c-e441e1b29c2b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:32.886749 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9f96-5477-4d97-952c-e441e1b29c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:32.905683 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-1f6e9f96-5477-4d97-952c-e441e1b29c2b None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:32.906861 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1f6e9f96-5477-4d97-952c-e441e1b29c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:32.963407 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1458/2941] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:32:32.972870 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:32.974299 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:33.032886 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f876fef-81c6-4086-8e21-fe4cc4eb4b1c admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.036590 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884264cc-b4a3-4945-bf8d-873b68ba4541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.037191 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884264cc-b4a3-4945-bf8d-873b68ba4541 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.037469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884264cc-b4a3-4945-bf8d-873b68ba4541 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.037781 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884264cc-b4a3-4945-bf8d-873b68ba4541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.046630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884264cc-b4a3-4945-bf8d-873b68ba4541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:33.048056 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884264cc-b4a3-4945-bf8d-873b68ba4541 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:33.059752 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1484/2942] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:33.318348 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e9f96-5477-4d97-952c-e441e1b29c2b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:33.319833 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-1f6e9f96-5477-4d97-952c-e441e1b29c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:33.332298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e9f96-5477-4d97-952c-e441e1b29c2b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.337138 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.337904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.338838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.338838 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] PATH_INFO: `/v3/projects/1f44713cf1b946ec980de91554b3988e` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.339712 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f44713cf1b946ec980de91554b3988e)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:33.341721 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:33.387984 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f44713cf1b946ec980de91554b3988e', 'request_id': 'req-cc506995-b033-45db-b442-5a7f026e6366'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:33.393409 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f44713cf1b946ec980de91554b3988e', 'request_id': 'req-cc506995-b033-45db-b442-5a7f026e6366'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:33.399523 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f44713cf1b946ec980de91554b3988e', 'request_id': 'req-cc506995-b033-45db-b442-5a7f026e6366'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:33.412450 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] The token cache is being invalidate because project 1f44713cf1b946ec980de91554b3988e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88442) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:32:33.412743 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:33.413365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-cc506995-b033-45db-b442-5a7f026e6366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:32:33.414881 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1459/2943] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:32:33 2022] DELETE /identity/v3/projects/1f44713cf1b946ec980de91554b3988e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:33.419850 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e23fe-50ce-4eba-b1ab-6426b4599c62 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.420135 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e23fe-50ce-4eba-b1ab-6426b4599c62 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.420407 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e23fe-50ce-4eba-b1ab-6426b4599c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.440157 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-4d2e23fe-50ce-4eba-b1ab-6426b4599c62 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:33.441386 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-4d2e23fe-50ce-4eba-b1ab-6426b4599c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:33.457887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:33.501198 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1485/2944] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:32:33.509037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287ba77-27c1-4977-b863-4d2fd81d7a2d None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.509356 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287ba77-27c1-4977-b863-4d2fd81d7a2d None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.509662 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287ba77-27c1-4977-b863-4d2fd81d7a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.529506 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-b287ba77-27c1-4977-b863-4d2fd81d7a2d None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:33.530742 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-b287ba77-27c1-4977-b863-4d2fd81d7a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:33.539172 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1460/2945] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:33.543503 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e23fe-50ce-4eba-b1ab-6426b4599c62 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:33.553782 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e23fe-50ce-4eba-b1ab-6426b4599c62 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.558326 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.558918 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.559195 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.559468 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] PATH_INFO: `/v3/domains` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.560107 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:32:33.560459 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:33.562139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:33.573439 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1486/2946] 10.222.0.23 () {58 vars in 1153 bytes} [Tue Sep 13 20:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:32:33.577390 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287ba77-27c1-4977-b863-4d2fd81d7a2d None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:33.586363 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287ba77-27c1-4977-b863-4d2fd81d7a2d None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.590990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2805f67-46d5-4d59-a3a8-55717b28726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.591574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2805f67-46d5-4d59-a3a8-55717b28726f admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.591853 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2805f67-46d5-4d59-a3a8-55717b28726f admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.592123 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2805f67-46d5-4d59-a3a8-55717b28726f admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.592738 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2805f67-46d5-4d59-a3a8-55717b28726f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:33.594699 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2805f67-46d5-4d59-a3a8-55717b28726f admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:33.622988 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1461/2947] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:32:33 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:33.626997 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:33.636954 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2a5a6-446c-4bf3-80bd-9610fd6f660d admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.693046 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e0a38-bca9-4b07-8b41-b1ff25f44768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.693673 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e0a38-bca9-4b07-8b41-b1ff25f44768 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.693952 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e0a38-bca9-4b07-8b41-b1ff25f44768 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.694232 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e0a38-bca9-4b07-8b41-b1ff25f44768 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.694855 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e0a38-bca9-4b07-8b41-b1ff25f44768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:33.696776 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e0a38-bca9-4b07-8b41-b1ff25f44768 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:33.719977 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1487/2948] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:32:33 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:32:33.724634 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2805f67-46d5-4d59-a3a8-55717b28726f admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:33.735037 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2805f67-46d5-4d59-a3a8-55717b28726f admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.739568 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072987d-12a0-433f-a532-f7ad3a0330f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.740171 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072987d-12a0-433f-a532-f7ad3a0330f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.740453 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072987d-12a0-433f-a532-f7ad3a0330f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.740729 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072987d-12a0-433f-a532-f7ad3a0330f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.741391 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072987d-12a0-433f-a532-f7ad3a0330f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:33.743219 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072987d-12a0-433f-a532-f7ad3a0330f5 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:33.753292 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1462/2949] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:33.757337 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e0a38-bca9-4b07-8b41-b1ff25f44768 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:33.767328 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e0a38-bca9-4b07-8b41-b1ff25f44768 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.772501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a421affa-6c44-4032-a7b7-b4cec9b1a023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.773089 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421affa-6c44-4032-a7b7-b4cec9b1a023 admin admin] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.773369 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421affa-6c44-4032-a7b7-b4cec9b1a023 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.773662 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a421affa-6c44-4032-a7b7-b4cec9b1a023 admin admin] PATH_INFO: `/v3/roles` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.774298 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421affa-6c44-4032-a7b7-b4cec9b1a023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:33.775882 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a421affa-6c44-4032-a7b7-b4cec9b1a023 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:33.787129 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1488/2950] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:33.791173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072987d-12a0-433f-a532-f7ad3a0330f5 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:33.801040 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072987d-12a0-433f-a532-f7ad3a0330f5 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.805583 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.806249 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.806529 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.806805 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] PATH_INFO: `/v3/projects/d255544711d242ca885e574b9170703b/users/6872a9b421bb4c19865b769827bc01ae/roles/afc24e28d6c34708ab897fc4b7029bb2` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.821035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d255544711d242ca885e574b9170703b, user_id=6872a9b421bb4c19865b769827bc01ae, role_id=afc24e28d6c34708ab897fc4b7029bb2)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:33.823670 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:33.836715 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1463/2951] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:32:33 2022] PUT /identity/v3/projects/d255544711d242ca885e574b9170703b/users/6872a9b421bb4c19865b769827bc01ae/roles/afc24e28d6c34708ab897fc4b7029bb2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:33.841474 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6c87d-caa0-47df-b7f6-ee3edcc6a5e2 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.841775 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6c87d-caa0-47df-b7f6-ee3edcc6a5e2 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.842049 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6c87d-caa0-47df-b7f6-ee3edcc6a5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:33.861630 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-aff6c87d-caa0-47df-b7f6-ee3edcc6a5e2 None None] MFA Rules not processed for user `6a5ebebe92dd4d8b9d923b7a40d319f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:33.862840 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-aff6c87d-caa0-47df-b7f6-ee3edcc6a5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:33.920110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1489/2952] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:33.929482 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:33.931192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:33.952426 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:33.990468 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b42a2e-e47e-4199-b798-c9a9045eee90 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:33.994092 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fc094-e573-4550-9c1a-b2bbda82836c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:33.994691 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc094-e573-4550-9c1a-b2bbda82836c service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:33.994975 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc094-e573-4550-9c1a-b2bbda82836c service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:33.995252 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc094-e573-4550-9c1a-b2bbda82836c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:34.004151 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fc094-e573-4550-9c1a-b2bbda82836c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:34.005581 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fc094-e573-4550-9c1a-b2bbda82836c service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:34.019047 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1464/2953] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:37.906922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e78d60-3489-42d4-acca-95fd7e7e3ad6 None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:37.907450 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e78d60-3489-42d4-acca-95fd7e7e3ad6 None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:37.907562 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e78d60-3489-42d4-acca-95fd7e7e3ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:37.936935 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-53e78d60-3489-42d4-acca-95fd7e7e3ad6 None None] MFA Rules not processed for user `6872a9b421bb4c19865b769827bc01ae`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:37.938205 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-53e78d60-3489-42d4-acca-95fd7e7e3ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:37.955452 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:38.004531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1490/2954] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:38.009991 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fc094-e573-4550-9c1a-b2bbda82836c service neutron] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:38.020770 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fc094-e573-4550-9c1a-b2bbda82836c service neutron] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:38.025378 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4dd2b-3563-409d-9fd4-fe795a8f0966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:38.026053 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4dd2b-3563-409d-9fd4-fe795a8f0966 admin admin] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:38.026283 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4dd2b-3563-409d-9fd4-fe795a8f0966 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:38.026547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4dd2b-3563-409d-9fd4-fe795a8f0966 admin admin] PATH_INFO: `/v3/projects` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:38.027231 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4dd2b-3563-409d-9fd4-fe795a8f0966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:38.029068 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4dd2b-3563-409d-9fd4-fe795a8f0966 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:38.075808 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1465/2955] 10.222.0.23 () {60 vars in 1153 bytes} [Tue Sep 13 20:32:38 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:32:38.079880 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e78d60-3489-42d4-acca-95fd7e7e3ad6 None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:38.093227 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e78d60-3489-42d4-acca-95fd7e7e3ad6 None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:38.152899 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab106f0-653a-4bd4-8b58-f0636db7bb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:38.153507 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab106f0-653a-4bd4-8b58-f0636db7bb47 admin admin] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:38.153822 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab106f0-653a-4bd4-8b58-f0636db7bb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:38.154090 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab106f0-653a-4bd4-8b58-f0636db7bb47 admin admin] PATH_INFO: `/v3/users` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:38.154727 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab106f0-653a-4bd4-8b58-f0636db7bb47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:38.156668 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab106f0-653a-4bd4-8b58-f0636db7bb47 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:38.184400 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1491/2956] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Sep 13 20:32:38 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:32:38.185636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4dd2b-3563-409d-9fd4-fe795a8f0966 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:38.196693 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4dd2b-3563-409d-9fd4-fe795a8f0966 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:38.201496 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265e050-5b42-4759-9c11-4d170b565545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:38.202136 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265e050-5b42-4759-9c11-4d170b565545 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:38.202443 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265e050-5b42-4759-9c11-4d170b565545 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:38.202717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265e050-5b42-4759-9c11-4d170b565545 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:38.203365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265e050-5b42-4759-9c11-4d170b565545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:38.204976 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265e050-5b42-4759-9c11-4d170b565545 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:38.216980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1466/2957] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:38.221266 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab106f0-653a-4bd4-8b58-f0636db7bb47 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:38.232110 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab106f0-653a-4bd4-8b58-f0636db7bb47 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:38.237204 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:38.238478 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] REQUEST_METHOD: `PUT` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:38.238758 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:38.239030 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] PATH_INFO: `/v3/projects/6f1a09c397854cc8b06ab65d83c80956/users/1ef77d61a95640bd9eca1383f6cfa65b/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:38.262163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1a09c397854cc8b06ab65d83c80956, user_id=1ef77d61a95640bd9eca1383f6cfa65b, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:38.264965 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:38.279139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1492/2958] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:32:38 2022] PUT /identity/v3/projects/6f1a09c397854cc8b06ab65d83c80956/users/1ef77d61a95640bd9eca1383f6cfa65b/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:32:38.282859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265e050-5b42-4759-9c11-4d170b565545 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:38.293365 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265e050-5b42-4759-9c11-4d170b565545 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:38.342341 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:38.344294 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:38.359980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199bca-888e-4a5a-bf11-b0d5d9689159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:38.360862 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199bca-888e-4a5a-bf11-b0d5d9689159 admin admin] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:38.361147 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199bca-888e-4a5a-bf11-b0d5d9689159 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:38.361421 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199bca-888e-4a5a-bf11-b0d5d9689159 admin admin] PATH_INFO: `/v3/roles` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:38.362117 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae199bca-888e-4a5a-bf11-b0d5d9689159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:38.364285 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae199bca-888e-4a5a-bf11-b0d5d9689159 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:38.378851 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1467/2959] 10.222.0.23 () {58 vars in 1122 bytes} [Tue Sep 13 20:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:32:38.380697 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199bca-888e-4a5a-bf11-b0d5d9689159 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:38.391235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199bca-888e-4a5a-bf11-b0d5d9689159 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:38.396545 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96b0e5-e826-4d85-818b-6a7197a9730c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:38.398077 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96b0e5-e826-4d85-818b-6a7197a9730c admin admin] REQUEST_METHOD: `PUT` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:38.398618 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96b0e5-e826-4d85-818b-6a7197a9730c admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:38.399043 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96b0e5-e826-4d85-818b-6a7197a9730c admin admin] PATH_INFO: `/v3/projects/6f1a09c397854cc8b06ab65d83c80956/users/1ef77d61a95640bd9eca1383f6cfa65b/roles/b698800ceec3460a8519ea627650da0f` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:38.401354 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96b0e5-e826-4d85-818b-6a7197a9730c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1a09c397854cc8b06ab65d83c80956, user_id=1ef77d61a95640bd9eca1383f6cfa65b, role_id=b698800ceec3460a8519ea627650da0f)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:38.404346 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96b0e5-e826-4d85-818b-6a7197a9730c admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:38.416772 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1468/2960] 10.222.0.23 () {60 vars in 1483 bytes} [Tue Sep 13 20:32:38 2022] PUT /identity/v3/projects/6f1a09c397854cc8b06ab65d83c80956/users/1ef77d61a95640bd9eca1383f6cfa65b/roles/b698800ceec3460a8519ea627650da0f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:32:38.421585 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c39dd5-d9b5-423d-a6e6-09ba6656cc40 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:38.450766 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:38.494073 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:38.494703 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:38.495008 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:38.495357 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:38.497416 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:38.514987 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:38.516616 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:38.556719 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1493/2961] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:32:42.940088 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5e54-cefb-4825-abcd-ae19aecbaae2 None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:42.940594 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5e54-cefb-4825-abcd-ae19aecbaae2 None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:42.940690 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a5e54-cefb-4825-abcd-ae19aecbaae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:42.969457 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-a59a5e54-cefb-4825-abcd-ae19aecbaae2 None None] MFA Rules not processed for user `1ef77d61a95640bd9eca1383f6cfa65b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:42.970462 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-a59a5e54-cefb-4825-abcd-ae19aecbaae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:43.043206 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:43.049345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1469/2962] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:43.062165 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:43.063680 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:43.125392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968b861-f104-4c0f-9786-77b7555fdbf9 service placement] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:43.133392 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413447-9395-479e-b142-a674eb779080 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d78c1e1b6e46e3ba6271e6743d3c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:43.135568 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413447-9395-479e-b142-a674eb779080 service glance] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:43.136131 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413447-9395-479e-b142-a674eb779080 service glance] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:43.136785 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e413447-9395-479e-b142-a674eb779080 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:43.197497 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e413447-9395-479e-b142-a674eb779080 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:43.199093 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e413447-9395-479e-b142-a674eb779080 service glance] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:43.208710 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1494/2963] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:43.354874 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047e192-81c1-4a11-8dda-4b56a02187df None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:43.355157 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047e192-81c1-4a11-8dda-4b56a02187df None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:43.355424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047e192-81c1-4a11-8dda-4b56a02187df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:43.375964 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-f047e192-81c1-4a11-8dda-4b56a02187df None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:43.377153 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-f047e192-81c1-4a11-8dda-4b56a02187df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:43.432668 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1470/2964] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 20:32:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:43.496485 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413447-9395-479e-b142-a674eb779080 service glance] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:43.499211 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-5e413447-9395-479e-b142-a674eb779080 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:43.576862 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e413447-9395-479e-b142-a674eb779080 service glance] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:43.583028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417c46c-571c-4a12-b32e-aa1548b985ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:43.583645 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417c46c-571c-4a12-b32e-aa1548b985ae service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:43.583922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417c46c-571c-4a12-b32e-aa1548b985ae service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:43.584190 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417c46c-571c-4a12-b32e-aa1548b985ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:43.593194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417c46c-571c-4a12-b32e-aa1548b985ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:43.595088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417c46c-571c-4a12-b32e-aa1548b985ae service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:43.600498 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=100 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:43.609679 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1495/2965] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 20:32:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:32:52.451569 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d9c99-d8fa-4617-9256-29a0c36fcf7b None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:52.452070 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d9c99-d8fa-4617-9256-29a0c36fcf7b None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:52.452183 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d9c99-d8fa-4617-9256-29a0c36fcf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:52.472149 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-336d9c99-d8fa-4617-9256-29a0c36fcf7b None None] MFA Rules not processed for user `6872a9b421bb4c19865b769827bc01ae`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:52.472990 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-336d9c99-d8fa-4617-9256-29a0c36fcf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:52.485104 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1471/2966] 10.222.0.23 () {58 vars in 958 bytes} [Tue Sep 13 20:32:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:52.530795 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417c46c-571c-4a12-b32e-aa1548b985ae service swift] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:52.540013 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417c46c-571c-4a12-b32e-aa1548b985ae service swift] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:52.543612 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '17d78c1e1b6e46e3ba6271e6743d3c73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:52.544194 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:52.544518 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:52.544735 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:52.546472 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:52.617134 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:52.618595 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:52.629469 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1496/2967] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:52.846466 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d9c99-d8fa-4617-9256-29a0c36fcf7b None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:52.900323 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d9c99-d8fa-4617-9256-29a0c36fcf7b None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:52.903873 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6479b-ad53-483d-80aa-99c00b9252cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:52.904470 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6479b-ad53-483d-80aa-99c00b9252cb service neutron] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:52.904754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6479b-ad53-483d-80aa-99c00b9252cb service neutron] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:52.905029 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6479b-ad53-483d-80aa-99c00b9252cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:52.906754 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-50c6479b-ad53-483d-80aa-99c00b9252cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:52.916614 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6479b-ad53-483d-80aa-99c00b9252cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:52.918156 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6479b-ad53-483d-80aa-99c00b9252cb service neutron] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:52.926119 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1472/2968] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:53.238454 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:53.250244 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca9f08-ae58-4f9e-ab9f-b4ebda13f97d service glance] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:53.255183 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:53.255804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:53.256088 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:53.256355 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:53.258122 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:53.269868 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:53.271981 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:53.282447 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1497/2969] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:32:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:54.524682 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83777a1-7206-4511-a306-239d764c80bb None None] REQUEST_METHOD: `POST` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:54.525182 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83777a1-7206-4511-a306-239d764c80bb None None] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:54.525287 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83777a1-7206-4511-a306-239d764c80bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:54.545812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.auth.core [None req-d83777a1-7206-4511-a306-239d764c80bb None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88442) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:32:54.546938 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d83777a1-7206-4511-a306-239d764c80bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:32:54.554026 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1473/2970] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Sep 13 20:32:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:32:54.567218 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:32:54.579047 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbb81e-e63a-4f37-be46-750bcc536cf2 service placement] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:32:54.584816 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f1e3-ff34-492f-8145-89279222541c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:32:54.585539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f1e3-ff34-492f-8145-89279222541c service swift] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:32:54.585913 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f1e3-ff34-492f-8145-89279222541c service swift] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:32:54.586257 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f1e3-ff34-492f-8145-89279222541c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:32:54.594501 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f1e3-ff34-492f-8145-89279222541c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:32:54.596282 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f1e3-ff34-492f-8145-89279222541c service swift] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:32:54.604814 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1498/2971] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 20:32:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:33:04.540872 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=58 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:04.601064 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=46 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:09.343367 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83777a1-7206-4511-a306-239d764c80bb None None] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:09.345014 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-d83777a1-7206-4511-a306-239d764c80bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:09.363259 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83777a1-7206-4511-a306-239d764c80bb None None] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:09.369062 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:09.369698 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:09.369980 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:09.370253 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:09.372033 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:09.384319 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:09.386685 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:09.400904 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1474/2972] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:19.393940 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:19.936531 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f1e3-ff34-492f-8145-89279222541c service swift] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:19.946057 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f1e3-ff34-492f-8145-89279222541c service swift] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:19.949588 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7e8f6-e4de-4c66-9f9e-3f06659a9e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f77e57b2d96461c88e2fc33db22e22d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:19.950212 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7e8f6-e4de-4c66-9f9e-3f06659a9e59 service neutron] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:19.951028 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7e8f6-e4de-4c66-9f9e-3f06659a9e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:19.951302 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7e8f6-e4de-4c66-9f9e-3f06659a9e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:19.968239 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7e8f6-e4de-4c66-9f9e-3f06659a9e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:19.969696 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff7e8f6-e4de-4c66-9f9e-3f06659a9e59 service neutron] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:19.979433 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1499/2973] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:21.536746 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:21.550474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd9f47e-8dc2-4155-acef-98bf65f72d76 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:21.556982 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2981bb0558d34329a061a5f08149bba3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:21.557600 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:21.557903 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:21.558345 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:21.560834 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:21.573717 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:21.575330 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:21.588547 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1475/2974] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:29.975221 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=6 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:31.581977 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=6 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:43.206330 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8823d8-b922-45b3-b8f2-a14aa9db903d None None] REQUEST_METHOD: `POST` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:43.206823 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8823d8-b922-45b3-b8f2-a14aa9db903d None None] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:43.206928 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8823d8-b922-45b3-b8f2-a14aa9db903d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:43.239066 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.auth.core [None req-fb8823d8-b922-45b3-b8f2-a14aa9db903d None None] MFA Rules not processed for user `803b35616c55468eb7a0f72c9c86b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=88441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:43.240274 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fb8823d8-b922-45b3-b8f2-a14aa9db903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:43.247453 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1500/2975] 10.222.0.23 () {58 vars in 967 bytes} [Tue Sep 13 20:34:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:34:43.287859 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:43.302097 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d5c75-085f-49b8-a8e3-29028330ad58 service nova] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:43.307071 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c3425-1f2c-427b-ae7f-1fcdd066f995 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d93aa681e2a4427a61fe0bf472c60ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:43.307669 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3425-1f2c-427b-ae7f-1fcdd066f995 service swift] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:43.307951 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3425-1f2c-427b-ae7f-1fcdd066f995 service swift] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:43.308227 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3425-1f2c-427b-ae7f-1fcdd066f995 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:43.318572 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c3425-1f2c-427b-ae7f-1fcdd066f995 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:43.320213 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c3425-1f2c-427b-ae7f-1fcdd066f995 service swift] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:43.328173 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1476/2976] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Sep 13 20:34:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:34:46.414967 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8823d8-b922-45b3-b8f2-a14aa9db903d None None] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:46.416483 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-fb8823d8-b922-45b3-b8f2-a14aa9db903d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:46.433530 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8823d8-b922-45b3-b8f2-a14aa9db903d None None] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:46.439539 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:46.440261 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] REQUEST_METHOD: `GET` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:46.440639 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:46.440986 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:46.442915 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:46.455615 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:46.457153 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:46.468598 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1501/2977] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:49.740235 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c3425-1f2c-427b-ae7f-1fcdd066f995 service swift] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:49.831021 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c3425-1f2c-427b-ae7f-1fcdd066f995 service swift] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:49.836636 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:49.837298 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:49.837585 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:49.837917 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] PATH_INFO: `/v3/users/6872a9b421bb4c19865b769827bc01ae` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:49.839060 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6872a9b421bb4c19865b769827bc01ae)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:49.840995 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:49.895574 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6872a9b421bb4c19865b769827bc01ae'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:49.903524 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6872a9b421bb4c19865b769827bc01ae'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:49.904236 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6872a9b421bb4c19865b769827bc01ae'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:49.916460 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6872a9b421bb4c19865b769827bc01ae'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:49.924317 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6872a9b421bb4c19865b769827bc01ae'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:49.930305 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1477/2978] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:34:49 2022] DELETE /identity/v3/users/6872a9b421bb4c19865b769827bc01ae => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:34:50.098755 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:50.110632 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90521a68-613f-4c96-bf86-bb1690f0a310 service placement] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:50.176189 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:50.176922 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:50.177243 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:50.177621 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] PATH_INFO: `/v3/projects/d255544711d242ca885e574b9170703b` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:50.178804 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d255544711d242ca885e574b9170703b)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:50.180716 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:50.225715 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd255544711d242ca885e574b9170703b', 'request_id': 'req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.231166 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd255544711d242ca885e574b9170703b', 'request_id': 'req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.237690 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd255544711d242ca885e574b9170703b', 'request_id': 'req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.267377 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] The token cache is being invalidate because project d255544711d242ca885e574b9170703b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:34:50.267730 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.268295 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.269866 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1502/2979] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:34:50 2022] DELETE /identity/v3/projects/d255544711d242ca885e574b9170703b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:34:50.273852 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:50.275372 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:50.288605 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e0395-6cd4-476b-a4cb-e74e97d27e86 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:50.331540 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:50.344424 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:50.345038 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:50.345812 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:50.346086 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] PATH_INFO: `/v3/users/1ef77d61a95640bd9eca1383f6cfa65b` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:50.347220 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ef77d61a95640bd9eca1383f6cfa65b)` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:50.349143 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:50.395035 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ef77d61a95640bd9eca1383f6cfa65b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.405871 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ef77d61a95640bd9eca1383f6cfa65b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.406560 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ef77d61a95640bd9eca1383f6cfa65b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.419036 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ef77d61a95640bd9eca1383f6cfa65b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.425882 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.notifications [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ef77d61a95640bd9eca1383f6cfa65b'} {{(pid=88442) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.431722 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1478/2980] 10.222.0.23 () {58 vars in 1224 bytes} [Tue Sep 13 20:34:50 2022] DELETE /identity/v3/users/1ef77d61a95640bd9eca1383f6cfa65b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:34:50.604947 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] Authenticating user token {{(pid=88441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:50.606524 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.fernet_utils [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:50.621646 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4cdcd-6e4d-43d3-87a1-3835eab37d79 admin admin] Validating token access rules against request {{(pid=88441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:50.679980 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:50.691963 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a5ebebe92dd4d8b9d923b7a40d319f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61d39e602be54c45a9fb48df499e3bc7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:50.692605 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:50.692887 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:50.693163 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] PATH_INFO: `/v3/projects/6f1a09c397854cc8b06ab65d83c80956` {{(pid=88441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:50.694352 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f1a09c397854cc8b06ab65d83c80956)` {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:50.696381 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] RBAC: Authorization granted {{(pid=88441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:50.743931 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f1a09c397854cc8b06ab65d83c80956', 'request_id': 'req-3e65a3fe-daf2-4928-89ce-fe230017b0e4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.749396 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f1a09c397854cc8b06ab65d83c80956', 'request_id': 'req-3e65a3fe-daf2-4928-89ce-fe230017b0e4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.756139 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f1a09c397854cc8b06ab65d83c80956', 'request_id': 'req-3e65a3fe-daf2-4928-89ce-fe230017b0e4'} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.769029 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] The token cache is being invalidate because project 6f1a09c397854cc8b06ab65d83c80956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:34:50.769325 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.769972 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG keystone.notifications [None req-3e65a3fe-daf2-4928-89ce-fe230017b0e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:34:50.771448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: [pid: 88441|app: 0|req: 1503/2981] 10.222.0.23 () {58 vars in 1233 bytes} [Tue Sep 13 20:34:50 2022] DELETE /identity/v3/projects/6f1a09c397854cc8b06ab65d83c80956 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:35:00.428306 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:00.765448 dv-cinder-828805-6-iscsi devstack@keystone.service[88441]: DEBUG dbcounter [-] [88441] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:37:43.416488 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] Authenticating user token {{(pid=88442) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:37:43.417912 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:43.536192 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97264db9-5629-464e-ba32-ff47033e4240 admin admin] Validating token access rules against request {{(pid=88442) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:37:43.595474 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c290b7-de41-4aa9-9b7e-0a4c3b51114c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8957fc12ab74474f890b9188ba632fab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c5e1753942b401696ad633361fa21c9', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88442) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:37:43.596084 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c290b7-de41-4aa9-9b7e-0a4c3b51114c service placement] REQUEST_METHOD: `GET` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:37:43.596370 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c290b7-de41-4aa9-9b7e-0a4c3b51114c service placement] SCRIPT_NAME: `/identity` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:37:43.596644 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c290b7-de41-4aa9-9b7e-0a4c3b51114c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88442) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:37:43.598436 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.fernet_utils [None req-50c290b7-de41-4aa9-9b7e-0a4c3b51114c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88442) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:37:43.609385 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c290b7-de41-4aa9-9b7e-0a4c3b51114c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:37:43.610998 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c290b7-de41-4aa9-9b7e-0a4c3b51114c service placement] RBAC: Authorization granted {{(pid=88442) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:37:43.621320 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: [pid: 88442|app: 0|req: 1479/2982] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Sep 13 20:37:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:37:53.617045 dv-cinder-828805-6-iscsi devstack@keystone.service[88442]: DEBUG dbcounter [-] [88442] Writing DB stats keystone:SELECT=80 {{(pid=88442) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}